Estimators, bias and standard error

Statistics and estimators

A statistic is a quantity calculated from the sample alone (no unknown parameters), for example X̄ = ΣXi/n. As the sample is random, a statistic is a random variable with its own distribution, the sampling distribution.

An estimator T of a parameter θ is a statistic used to estimate θ; the value it takes for a particular sample is an estimate.

Bias

T is unbiased for θ if E(T) = θ. Otherwise bias = E(T) - θ.

To show unbiasedness, use expectation algebra with E(Xi) = μ and write a conclusion: 'E(T) = μ, so T is an unbiased estimator of μ'. Examiners deduct the final mark when the conclusion is missing, and when μ is left out of a bias expression.

Example: X1, X2, X3 is a random sample. T1 = (X1 + 2X2 + X3)/4 has E(T1) = (μ + 2μ + μ)/4 = μ: unbiased. T2 = (2X1 + X2 + X3)/3 has E(T2) = 4μ/3, so bias = μ/3.

Choosing the better estimator

Among unbiased estimators, the one with the smaller variance is better, because its values cluster more tightly about θ.

Var(T1) = (1 + 4 + 1)σ²/16 = 3σ²/8, while Var(X̄) = σ²/3. As 1/3 < 3/8, X̄ is the better estimator.

When unknown constants are involved, set E(T) = μ and cancel μ before solving.

Standard error

The standard deviation of an estimator is its standard error. For the sample mean, standard error = σ/sqrt(n), estimated by s/sqrt(n) if σ is unknown. Quadrupling the sample size halves the standard error.

Worked example

T = (X1 + aX2)/5 is an unbiased estimator of μ. Find a and Var(T) in terms of σ².

1. E(T) = (μ + aμ)/5 = μ, so (1 + a)/5 = 1 after cancelling μ.
2. a = 4.
3. Var(T) = (Var(X1) + 16Var(X2))/25 = 17σ²/25

Answer: a = 4, Var(T) = 17σ²/25

Check yourself

1. X1, X2, X3 is a random sample from a population with mean μ and variance σ². T1 = (X1 + 2X2 + X3)/4. Var(T1) = kσ². Find k. Give your answer as a fraction in its simplest form.
   Show answer

   3/8. Var(T1) = (1 + 4 + 1)σ²/16 = 6σ²/16 = 3σ²/8.

2. X1, X2, X3 is a random sample from a population with mean μ. T2 = (2X1 + X2 + X3)/3. The bias of T2 is kμ. Find k. Give your answer as a fraction in its simplest form.
   Show answer

   1/3. E(T2) = 4μ/3, so bias = 4μ/3 - μ = μ/3.

3. T = (X1 + aX2)/5 is an unbiased estimator of the population mean μ. Find a.
   Show answer

   4. (1 + a)μ/5 = μ, so 1 + a = 5 and a = 4.

4. T = (X1 + 4X2)/5, where X1 and X2 are independent with variance σ². Var(T) = kσ². Find k. Give your answer as a fraction in its simplest form.
   Show answer

   17/25. (1 + 16)σ²/25 = 17σ²/25.

5. A population has standard deviation 12. Find the standard error of the mean of a random sample of 36.
   Show answer

   2. σ/sqrt(n) = 12/6 = 2.

6. T1 and T2 are both unbiased estimators of μ, with Var(T1) = 3σ²/8 and Var(T2) = σ²/3. Which is better, and why?
   • T1, because it has the larger variance
   • T2, because it has the smaller variance
   • T1, because 3/8 is a simpler fraction
   • Neither is better, because each is unbiased
   Show answer

   T2, because it has the smaller variance. Smaller variance means estimates are more tightly clustered around μ.

7. What does it mean for T to be an unbiased estimator of θ?
   • Var(T) = 0
   • T = θ for every sample
   • E(T) = 0
   • E(T) = θ
   Show answer

   E(T) = θ. On average, over all possible samples, T equals θ.

8. μ is unknown. Which of these is a statistic?
   • X̄ - μ
   • μ/2
   • The sample mean X̄
   • (X1 - μ)²
   Show answer

   The sample mean X̄. A statistic can be calculated from the sample alone, with no unknown parameters.

Practise with instant feedback Print this lesson

Sources

• Pearson Edexcel International Advanced Level in Mathematics specification (issue 3), Unit S3
• Pearson Edexcel IAL Statistics S3 (WST03) examiners' report, June 2025

Last updated 8 November 2026. Spotted a mistake? Tell us.