Debates in diagnosis: reliability, validity and culture

Reliability

A diagnosis is reliable if different clinicians (inter-rater reliability) or the same clinician at different times (test-retest) reach the same diagnosis.

Reliability is measured with kappa: 1.0 is perfect agreement. In the DSM-5 field trials (Regier et al., 2013), major depressive disorder reached a kappa of only 0.28, in the range the researchers called 'questionable' (0.20 to 0.39); schizophrenia reached 0.46 ('good').

Why reliability can be low: symptoms are reported verbally, clinicians interpret them differently, and patients present differently on different days.

Validity

A diagnosis is valid if it identifies a real, distinct disorder.

• Aetiological validity: people with the same diagnosis share the same cause.
• Predictive validity: the diagnosis predicts outcome and response to treatment.
• Descriptive validity: people with different diagnoses differ in their symptoms.

Problems: comorbidity (many people meet criteria for more than one disorder) and symptom overlap (for example, low motivation appears in depression and in the negative symptoms of schizophrenia). Rosenhan (1973) found that staff could not tell sane pseudo-patients from patients, which questions validity.

Cultural issues

Culture affects how distress is expressed, how clinicians interpret it, and whether help is sought.

• Hearing voices or seeing visions may be valued in some religious contexts but read as hallucinations in another.
• A clinician from a different culture may misread behaviour (for example, avoiding eye contact) as a symptom.
• Disorders can be culturally specific: DSM-5 includes a glossary of cultural concepts of distress.
• Suzuki et al. (2014) found that Japanese inpatients with schizophrenia were often underweight, unlike the Western picture of weight gain on antipsychotics, showing features differ across cultures.

Counter-argument: DSM and ICD are regularly revised, and ICD is written for international use, which reduces bias (the June 2025 examiners credited this).

Worked example

A clinician in the UK diagnoses a recently arrived patient with schizophrenia because she says she hears her late grandmother's voice. Discuss why culture may affect this diagnosis.

1. In the patient's culture, hearing a dead relative may be a normal part of grieving or religious belief.
2. A clinician unfamiliar with that culture may interpret it as a hallucination, so the diagnosis may lack validity.
3. Language barriers may make it harder to assess other symptoms, lowering reliability.
4. However, DSM-5 asks clinicians to consider cultural context, which should reduce this risk.

Answer: Culture can lead to misdiagnosis, but cultural guidance in modern manuals aims to reduce it.

Check yourself

1. What does inter-rater reliability of diagnosis mean?
   • Different clinicians give the same diagnosis to the same patient
   • A diagnosis predicts the outcome of treatment
   • People with a diagnosis share the same cause
   • Patients agree with their diagnosis
   Show answer

   Different clinicians give the same diagnosis to the same patient. Inter-rater reliability is agreement between clinicians.

2. A diagnosis that accurately predicts how a patient will respond to treatment has good
   • inter-rater reliability
   • test-retest reliability
   • predictive validity
   • ecological validity
   Show answer

   predictive validity. Predictive validity is about forecasting outcome.

3. Aetiological validity refers to whether people with the same diagnosis
   • share the same cause
   • live in the same country
   • receive the same drug
   • have the same clinician
   Show answer

   share the same cause. Aetiology means cause.

4. In the DSM-5 field trials, major depressive disorder had a kappa of 0.28. On a scale where 1.0 is perfect agreement, how far short of perfect is this? Give your answer to 2 decimal places.
   Show answer

   0.72. 1.00 - 0.28 = 0.72.

5. Why does comorbidity challenge the validity of diagnosis?
   • It shows diagnosis is always reliable
   • It suggests the categories may not describe separate disorders
   • It proves disorders are caused by culture
   • It means drugs never work
   Show answer

   It suggests the categories may not describe separate disorders. If many people meet criteria for several disorders, the boundaries between categories may not be real.

6. Explain one reason why culture may influence the diagnosis of a mental disorder.
   Show answer

   Clinicians may misinterpret behaviour that is normal in the patient's culture, such as hearing a dead relative's voice, as a symptom, so the patient may be wrongly diagnosed.. Behaviour normal in one culture can be read as a symptom in another.

Practise with instant feedback Print this lesson

Sources

• Pearson Edexcel International Advanced Subsidiary/Advanced Level in Psychology specification (issue January 2026)
• Pearson IAL Psychology component guide: WPS04 (Issue 2, November 2024)
• Regier et al. (2013), DSM-5 field trials in the United States and Canada, Part II: test-retest reliability of selected categorical diagnoses, American Journal of Psychiatry
• Pearson IAL Psychology summary of studies: WPS04
• Pearson WPS04/01 examiners' report, June 2025

Last updated 8 November 2026. Spotted a mistake? Tell us.