Generating Functions for Bernoulli-based Distributions

For the two uniform distributions and basic Bernoulli, there is really nothing much that can be done as n (or a and b) vary. However, for all distributions that have the opportunity to "scale" larger and larger, we will first determine the M(t) and then demonstrate the a surprising relationship as their variables to grow appropriately.

🔗

Theorem 7.5.4. Moment Generating Function for Binomial.

🔗

M (t) = {(p e^{t} + (1 - p))}^{n}

🔗

Proof.

\begin{aligned} M (t) & = \sum_{x = 0}^{n} e^{t x} (\binom{n}{x}) p^{x} (1 - p)^{n - x} \\ = \sum_{x = 0}^{n} (\binom{n}{x}) (p e^{t})^{x} (1 - p)^{n - x} \\ = {(p e^{t} + (1 - p))}^{n} \end{aligned}

where we used the binomial theorem to simplify the sum.

🔗

Notice that the moment generating function for Bernoulli is simply the Binomial moment generating function with n=1.

🔗

Corollary 7.5.5. Binomial Properties via Moment Generating Function.

🔗

M (0) = 1

M^{'} (0) = n p

M^{″} (0) = n p (1 - p) + (n p)^{2}

🔗

Proof.

M (0) = {(p e^{0} + (1 - p))}^{n} = 1^{n} = 1.

Taking the derivative with respect to t,

M^{'} (t) = n {(p e^{t} + (1 - p))}^{n - 1} p e^{t}

and evaluating at t=0 gives

M^{'} (0) = n {(p + (1 - p))}^{n - 1} p = n 1^{n - 1} p = n p .

Again, taking another derivative with respect to t,

M^{″} (t) = n (n - 1) {(p e^{t} + (1 - p))}^{n - 2} p^{2} e^{2 t} + n {(p e^{t} + (1 - p))}^{n - 1} p e^{t}

and evaluating at t=0 gives

\begin{aligned} M^{″} (0) & = n (n - 1) (p + (1 - p))^{n - 2} p^{2} + n (p + (1 - p))^{n - 1} p \\ = n (n - 1) p^{2} + n p = (n p)^{2} + n p - n p^{2} = n p (1 - p) + (n p)^{2} . \end{aligned}

🔗

Theorem 7.5.6. Moment Generating Function for Negative Binomial.

🔗

M (t) = \frac{p^{r}}{(1 - e^{t} (1 - p))^{r}}

🔗

Proof.

Using the a previous theorem 7.4.5 justifying the Negative Binomial probability function with

a = p e^{t}

and

b = 1 - p

and by changing variables to

u = x - r

gives

\begin{aligned} M (t) & = \sum_{u = 0}^{\infty} e^{t u} (\binom{u + r - 1}{r - 1}) (1 - p)^{u} p^{r} \\ = p^{r} \sum_{u = 0}^{\infty} (\binom{u + r - 1}{r - 1}) (e^{t} (1 - p))^{u} \\ = \frac{p^{r}}{(1 - e^{t} (1 - p))^{r}} \sum_{u = 0}^{\infty} (\binom{u + r - 1}{r - 1}) (1 - e^{t} (1 - p))^{r} (e^{t} (1 - p))^{u} \\ = \frac{p^{r}}{(1 - e^{t} (1 - p))^{r}} \end{aligned}

noting that the last summation is the the sum of a negative binomial probability function over its entire range.

It should be noted one may also rewrite the summation and appeal directly to the Negative Binomial Series 7.4.1 to also prove this result.

🔗

Notice that the moment generating function for Geometric is simply the Negative Binomial moment generating function with r=1.

🔗

Corollary 7.5.7. Negative Binomial Properties via Moment Generating Function.

🔗

M (0) = 1

M^{'} (0) = \frac{r}{p}

M^{″} (0) = {(\frac{r (1 - p)}{p^{2}})}^{2} + {(\frac{r}{p})}^{2}

🔗

Proof.

M (0) = \frac{(p)^{r}}{(1 - (1 - p))^{r}} = \frac{p^{r}}{p^{r}} = 1.

Taking the derivative with respect to t,

M^{'} (t) = - \frac{{(p e^{t} - e^{t} + 1)}^{r - 1} (p - 1) p^{r} r e^{t}}{{(p e^{t} - e^{t} + 1)}^{2 r}}

and evaluating at t=0 gives

M^{'} (0) = - \frac{p^{r - 1} p - 1 p^{r} r}{p^{2 r}} = \frac{r (1 - p)}{p} .

OOPS...don’t need the (1-p) on top. Again, taking another derivative with respect to t,

M^{″} (t) = - \frac{(p e^{t} - e^{t}) {(p e^{t} - e^{t} + 1)}^{r - 2} (p - 1) p^{r} (r - 1) r e^{t}}{{(p e^{t} - e^{t} + 1)}^{2 r}} + \frac{2 (p e^{t} - e^{t}) {(p e^{t} - e^{t} + 1)}^{2 r - 2} (p - 1) p^{r} r^{2} e^{t}}{{(p e^{t} - e^{t} + 1)}^{3 r}} - \frac{{(p e^{t} - e^{t} + 1)}^{r - 1} (p - 1) p^{r} r e^{t}}{{(p e^{t} - e^{t} + 1)}^{2 r}}

and evaluating at t=0 gives

\begin{aligned} M^{″} (0) & = \frac{(p^{2 r - 1} r - p^{2 r - 2} r - p^{2 r - 2}) (p - 1) r}{p^{2 r}} \\ = {(\frac{r (1 - p)}{p^{2}})}^{2} + {(\frac{r}{p})}^{2} . \end{aligned}

🔗

Just in case you are wondering about the derivations and especially taking the derivatives above, Sage will do the heavy lifting for you.


    
        
xxxxxxxxxx
 
1
var('t,x,n,p,mu,r')
2
assume(p < 1)
3
assume(p > 0)
4
f = binomial(x+r-1,r-1)*p^r*(1-p)^x
5
M = sum(e^(t*x)*f,x,0,infinity).factor()
6
show(M)
7
show(M(t=0))
8
Mt = derivative(M,t).factor()
9
show(Mt(t=0).simplify())
10
Mtt = derivative(Mt,t)
11
show(Mtt)
12
show(Mtt(t=0).simplify())
13
show((Mtt(t=0)-Mt(t=0)^2).factor())

    
    
    
    
        
            
                Language:
                
            
        
    
    




    
    
        
        Messages

Essentials of Mathematical Probability and Statistics

Search Results:

Section 7.5 Generating Functions for Bernoulli-based Distributions

Theorem 7.5.1. Moment Generating Function for Bernoulli.

Proof.

Theorem 7.5.2. Moment Generating Function for Geometric.

Proof.

Corollary 7.5.3. Geometric Properties via Moment Generating Function.

Proof.

Theorem 7.5.4. Moment Generating Function for Binomial.

Proof.

Corollary 7.5.5. Binomial Properties via Moment Generating Function.

Proof.

Theorem 7.5.6. Moment Generating Function for Negative Binomial.

Proof.

Corollary 7.5.7. Negative Binomial Properties via Moment Generating Function.

Proof.