Efficiency (statistics)

In statistics, statistical efficiency is one measure of desirability of an estimator.

The efficiency of an unbiased statistic T is defined as

where I(&theta) is the Fisher information of the sample. Thus e(T) is the minimum possible variance for an unbiased estimator divided by its actual variance. The Cramér-Rao inequality proves that e(T) ≤ 1.

Examples

Consider a sample of size n drawn from a normal distribution of mean &mu and unit variance.

The sample mean of the sample , defined as

has variance 1/n. This is equal to the Fisher information from the sample (this is clear from the definition) and thus the sample mean is efficient in the sense that its efficiency is one.

Now consider the sample median. This is an unbiased and consistent estimator for &mu. For large n the sample median is approximately normally distributed with mean &mu and variance &pi/(2n). The efficiency is thus 2/&pi, or about 64%. Note that this is the asymptotic efficiency---that is, the efficiency in the limit as sample size n tends to infinity. For finite values of n the efficiency is higher than this (for example, a sample size of 3 gives an efficiency of about 74%).

Many workers prefer the sample median as an estimator of the mean, holding that the loss in efficiency is more than compensated for by its enhanced robustness in terms of its insensitivity to outliers.

Relative efficiency

If and are estimators for the parameter θ, then most people would agree that T is "more efficient" than T ′ if: (i) its mean squared error is smaller for at least some value of , and (ii) the MSE does not exceed that of T ′ for any value of θ.

Formally,

holds for all , with strict inequality holding somewhere.

The relative efficiency would be defined as

Although e is in general a function of θ, in many cases the dependence drops out; if this is so, e being less than one would indicate that T is preferable, whatever the true value of θ.