Central Limit Theorem

Section 9.7 Central Limit Theorem

Often, when one wants to solve various scientific problems, several assumptions will be made regarding the nature of the underlying setting and base their conclusions on those assumptions. Indeed, if one is going to use a Binomial Distribution or a Negative Binomial Distribution, an assumption on the value of p is necessary. For Poisson and Exponential Distributions, one must know the mean. For Normal Distributions, one must assume values for both the mean and the standard deviation. Where do these values come from? Often, one may perform a preliminary study and obtain a sample statistic...such as a sample mean or a relative frequency and use these values for μ or p.

🔗

But what is the underlying distribution of these sample statistics? The Central Limit Theorem gives the answer...

🔗

The results from the previous section illustrate the tendency for bell-shaped distributions. This tendency can be described more mathematically through the following theorem. It is presented here without proof.

🔗

Theorem 9.7.1. Central Limit Theorem.

🔗

Presume X is a random variable from a distribution with known mean

μ

and known variance

σ_{x}^{2} .

For some natural number n, sample the distribution repeatedly creating a string of random variables denoted

X_{1}, X_{2}, . . ., X_{n}

and set

\overset{―}{X} = \frac{\sum X_{k}}{n} .

🔗

Then,

\overset{―}{X}

is approximately normally distributed with mean

μ

and variance

σ^{2} = \frac{σ_{x}^{2}}{n} .

🔗

Often the Central Limit Theorem is stated more formally using a conversion to standard units. Indeed, the theorem indicates that the random variable

\overset{―}{X}

has variance

\frac{σ^{2}}{n}

which means as n grows this variance approaches 0. So, the limiting random variable has a zero variance and therefore is no longer a random variable. To avoid this issue, the Central Limit Theorem is often stated as:

🔗

For random variables

W_{n} = \frac{\overset{―}{X} - μ}{σ / \sqrt{n}}

🔗

with corresponding distribution function

F_{n} (W_{n}),

lim_{n \to \infty} F_{n} (c) = \int_{- \infty}^{c} \frac{1}{\sqrt{2 π}} e^{- z^{2} / 2} d z = Φ (c)

🔗

that is, the standard normal distribution function.

🔗

While we are at it, we can again "go backwards" and figure out the mean and variance if given some probabilities.

🔗

Checkpoint 9.7.2. WebWork - Computing probabilities on $\overset{―}{X}$ .

A sample of

n = 11

observations is drawn from a normal population with

μ = 920

and

σ = 250 .

Find each of the following:

P (\bar{X} > 1055)

Probability =

P (\bar{X} < 791)

Probability =

P (\bar{X} > 844)

Probability =

Answer 1.

0.0366485

Answer 2.

0.043506

Answer 3.

0.843334

🔗

Example 9.7.3. Exponential X vs Normal $\overset{―}{X}$ .

Consider an exponential variable X with mean time till first success of

μ = 4 .

Then,

σ = 2

using the exponential formulas.

You can use the exponential probability function to compute probabilities dealing with X. Indeed,

P (X < 3.9) = F (3.9) = 1 - e^{- 3.9 / 4} \approx 0.6228 .

If instead you plan to sample from this distribution n=32 times, the Central Limit Theorem implies that you will get a random variable

\overset{―}{X}

which has an approximate normal distribution with the same mean but with new variance

σ_{\overset{―}{X}}^{2} = \frac{4}{32} = \frac{1}{8} .

Therefore

P (\overset{―}{X} < 3.9) \approx normalcdf (0, 3.9, 4, s q r t (1 / 8)) = 0.3886 .

🔗

When converting probability problems from continuous (such as exponential or uniform) then no adjustment to the question is needed since you are approximating one area with another area. However, when converting probability problems from discrete (such as binomial or geometric) then you need to consider how the interval would need to be adjusted so that histogram areas for the discrete problem would relate to areas under the normal curve. Generally, you will need to expand the stated interval each way by 1/2.

🔗

The Central Limit Theorem provides that regardless of the distribution of X, the distribution of an average of X’s is approximately normally distributed. However, it also shows why X may also be approximated for some distributions using the normal distribution as certain parameters are allowed to increase. Below, you can see how Binomial and Poisson distributions can be approximated directly using the Normal distribution.

🔗

Toward that end, for

0 < p < 1

consider a sequence of Bernoulli trials

Y_{1}, Y_{2}, . . ., Y_{n}

with each over the space {0,1}. Then,

X = \sum_{k = 1}^{n} Y_{k}

🔗

is a Binomial variable.

🔗

Theorem 9.7.4. The Binomial Distribution approximately Normal if n large..

🔗

Given a Binomial variable X with

μ = n p

and

σ^{2} = n p (1 - p),

then X is approximately also normal with the same mean and variance so long as

n p > 5

and

n (1 - p) > 5 .

🔗

Proof.

Using the Bernoulli variables

Y_{k}

each with mean p and variance p(1-p), note that the Central Limit Theorem applied to

\overset{―}{X} = \frac{\sum Y_{k}}{n}

gives that

\frac{\overset{―}{X} - p}{\sqrt{p (1 - p) / n}}

is approximately standard normal. By multiplying top and bottom by n yields

\frac{\sum Y_{k} - n p}{\sqrt{n p (1 - p)}}

is approximately standard normal. But

\sum Y_{k}

actually is the sum of the number of successes in n trials and is therefore a Binomial variable.

🔗

Example 9.7.5. Binomial as Normal.

Binomial becomes normal as

n \to \infty .

Consider n = 50 and p = 0.3. Then,

μ = 15

and

σ^{2} = 10.5 .

Using the binomial formulas, for example,

P (X = 16) = (\binom{50}{16}) {0.3}^{16} \cdot {0.7}^{34} \approx 0.11470

Using the normal distribution,

\begin{aligned} P (X = 16) & = P (15.5 < X < 16.5) \\ \approx n o r m a l c d f (15.5, 16.5, 15, s q r t (10.5)) \\ = 0.11697 \end{aligned}

Notice that these are very close.

🔗

Corollary 9.7.6. The Poisson Distribution is approximately normal with $μ$ large..

🔗

Given a Poisson variable X with

μ

and

σ^{2} = μ

given, then X is approximately also normal with the same mean and variance so long as

μ > 5 .

🔗

Proof.

Note from before that the Poisson distribution function was derived by approximating with Binomial and letting n approach infinity. Therefore, by the previous theorem, the Poisson variable is also approximately Normal using the Poisson mean and variance rather than the binomial’s. Indeed, in standard units

\frac{Y - μ}{\sqrt{μ}}

is approximately normal for large

μ .

🔗

Example 9.7.7. Poisson as Normal.

Poisson becomes normal as

μ \to \infty .

Consider

μ = 20 .

Then,

σ^{2} = μ = 20 .

Using the Poisson formulas, for example,

P (X = 19) = \frac{20^{19} e^{- 20}}{19!} \approx 0.08883

Using the normal distribution,

\begin{aligned} P (X = 19) & = P (18.5 < X < 19.5) \\ \approx n o r m a l c d f (18.5, 19.5, 20, s q r t (20)) \\ = 0.08683 \end{aligned}

Again, these are very close.

🔗

Example 9.7.8. Gamma as Normal.

Gamma becomes normal as

r \to \infty .

Assume that the average time till a first success is 12 minutes and that

r = 8 .

Then, the mean for the Gamma distribution is

μ = 12 \cdot 8 = 96

and

σ^{2} = 8 \cdot 12^{2} = 1152

and so

σ \approx 33.9411 .

Using the Gamma formulas,

\begin{aligned} P (90 \leq X \leq 100) & = \int_{90}^{100} f (x) d x \\ = 0.59252 - 0.47536 = 0.11716 . \end{aligned}

Using the normal distribution,

P (90 \leq X \leq 100) \approx n o r m a l c d f (90, 100, 96, 33.9411) = 0.11707 .

Amazingly, these are also very close.

🔗

Example 9.7.9. Uniform X vs Normal $\overset{―}{X}$ .

Consider a discrete uniform variable X over R = {1,2,...,20}. Then,

μ = 10.5

and

σ = \frac{20^{2} - 1^{2}}{20}

using the uniform formulas.

You can use the uniform probability function to compute probabilities dealing with X. Indeed,

P (8 \leq X < 12) = P (X \in {8, 9, 10, 11} = \frac{4}{20} = 1 / 5.

If instead you plan to sample from this distribution n=49 times, the Central Limit Theorem implies that you will get a random variable

\overset{―}{X}

which has an approximate normal distribution with the same mean but with new variance

σ_{\overset{―}{X}}^{2} = \frac{199 / 20}{49} = \frac{199}{580} .

Therefore, expanding the interval to include the boundaries of the corresponding histogram areas,

P (8 \leq \overset{―}{X} < 12) = P (7.5 \leq \overset{―}{X} \leq 11.5) \approx n o r m a l c d f (7.5, 11.5, 10.5, 0.585750) \approx 0.9561 .

🔗

As these examples illustrate, you will have increasing success in approximating the desired probabilities so long as the distribution’s corresponding parameter is allowed to be "sufficiently large". The mathematical reasoning this is true is not provided but depends upon the "Central Limit Theorem" discussed in the next section.

🔗

The above theorems allow you to utilize the normal distribution to compute approximate probabilities for the variable X in the stated distributions. This is not always true for all distributions since some do not have parameters which allow for approaching normality. However, regardless of the distribution the Central Limit Theorem always allows you to approximate probabilities if they involve an average of repeated attempts...that is, for variable

\overset{―}{X} .

This usefulness is illustrated in the examples below.

Essentials of Mathematical Probability and Statistics

Search Results:

Section 9.7 Central Limit Theorem

Theorem 9.7.1. Central Limit Theorem.

Checkpoint 9.7.2. WebWork - Computing probabilities on $\overset{―}{X}$ .

Example 9.7.3. Exponential X vs Normal $\overset{―}{X}$ .

Theorem 9.7.4. The Binomial Distribution approximately Normal if n large..

Proof.

Example 9.7.5. Binomial as Normal.

Corollary 9.7.6. The Poisson Distribution is approximately normal with $μ$ large..

Proof.

Example 9.7.7. Poisson as Normal.

Example 9.7.8. Gamma as Normal.

Example 9.7.9. Uniform X vs Normal $\overset{―}{X}$ .

Section 9.7 Central Limit Theorem

Theorem 9.7.1. Central Limit Theorem.

Checkpoint 9.7.2. WebWork - Computing probabilities on X―.

Example 9.7.3. Exponential X vs Normal X―.

Theorem 9.7.4. The Binomial Distribution approximately Normal if n large..

Proof.

Example 9.7.5. Binomial as Normal.

Corollary 9.7.6. The Poisson Distribution is approximately normal with μ large..

Proof.

Example 9.7.7. Poisson as Normal.

Example 9.7.8. Gamma as Normal.

Example 9.7.9. Uniform X vs Normal X―.

Checkpoint 9.7.2. WebWork - Computing probabilities on $\overset{―}{X}$ .

Example 9.7.3. Exponential X vs Normal $\overset{―}{X}$ .

Corollary 9.7.6. The Poisson Distribution is approximately normal with $μ$ large..

Example 9.7.9. Uniform X vs Normal $\overset{―}{X}$ .