Theorem 9.5.1. Moment Generating Function for Normal.
Presuming \(t \gt 0\) and
\begin{equation*}
M(t) = e^{t \mu+\frac{1}{2}t^2\sigma^2}
\end{equation*}
Section 9.5 Generating Functions for Normal and Associated Distributions
Moment Generating Functions 5.5.1 can be derived for each of the distributions in this chapter.
Proof.
Using the Normal probability function 9.2.1,
\begin{align*}
M(t) & = \int_{-\infty}^{\infty} e^{tx} \frac{1}{\sigma \sqrt{2 \pi}} e^{ -\left ( \frac{x-\mu}{\sigma} \right ) ^2 / 2} dx\\
& = \int_{-\infty}^{\infty} e^{t(z \sigma + \mu)} \frac{1}{\sqrt{2 \pi}} e^{ -z ^2 / 2} dz\\
& = e^{t \mu} \int_{-\infty}^{\infty} \frac{1}{\sqrt{2 \pi}} e^{ -(z ^2 - 2t z \sigma ) / 2} dz\\
& = = e^{t \mu} \int_{-\infty}^{\infty} \frac{1}{\sqrt{2 \pi}} e^{ -(z ^2 - 2t z \sigma + t^2 \sigma^2 - t^2 \sigma^2 ) / 2} dz\\
& = e^{t \mu+\frac{1}{2}t^2\sigma^2} \int_{-\infty}^{\infty} \frac{1}{\sqrt{2 \pi}} e^{ -\left (z - t \sigma \right )^2 / 2} dz\\
& = e^{t \mu+\frac{1}{2}t^2\sigma^2} \cdot 1
\end{align*}
where the final integral is just a shifted standard normal and therefore has value 1.
Corollary 9.5.2. Normal Properties via Moment Generating Function.
For the Normal variable X,
\begin{equation*}
M(0) = 1
\end{equation*}
\begin{equation*}
M'(0) = \mu
\end{equation*}
\begin{equation*}
M''(0) = \sigma^2 + \mu^2
\end{equation*}
Proof.
\begin{equation*}
M(0) = e^{0 \mu+\frac{1}{2}0^2\sigma^2} = e^0 = 1.
\end{equation*}
Continuing,
\begin{equation*}
M'(t) = {\left(\sigma^2 t + \mu \right)} e^{\left(\frac{1}{2} \sigma^2 t^{2} + \mu t \right)}
\end{equation*}
and therefore
\begin{equation*}
M'(0) = {\left(\sigma^2 0 + \mu \right)} e^{\left(\frac{1}{2} \sigma^2 0^{2} + \mu 0 \right)} = \mu e^0 = \mu.
\end{equation*}
Continuing with the second derivative,
\begin{equation*}
M''(t) = {\left(\sigma^2 t + \mu\right)}^2 e^{\left(\frac{1}{2} \sigma^2 t^2 + \mu t\right)} + \sigma^2 e^{\left(\frac{1}{2} \sigma^2 t^2 + \mu t\right)}
\end{equation*}
and therefore
\begin{equation*}
M''(0) = {\left(\sigma^2 0 + \mu\right)}^2 e^{\left(\frac{1}{2} \sigma^2 0^2 + \mu 0\right)} + \sigma^2 e^{\left(\frac{1}{2} \sigma^2 0^2 + \mu 0\right)} = \mu^2 + \sigma^2
\end{equation*}
which is the squared mean plus the variance for the normal distribution.
Theorem 9.5.3. Moment Generating Function for Chi-Square.
Presuming \(t \gt 0\) and
\begin{equation*}
M(t) = \left ( \frac{1}{1-2t} \right )^{r/2}
\end{equation*}
Proof.
Using the Chi-Square probability function 9.3.1,
\begin{align*}
M(t) & = \int_0^{\infty} e^{tx} \frac{x^{r/2-1} e^{-x/2} }{\Gamma(r/2) 2^{r/2} } dx\\
& = \int_0^{\infty} \frac{x^{r/2-1} e^{-x(1-2 t)/2} }{\Gamma(r/2) 2^{r/2}} dx\\
& = \int_0^{\infty} \frac{\left ( \frac{u}{1-2t} \right )^{r/2-1} e^{-u/2} }{\Gamma(r/2) 2^{r/2} } \frac{1}{1-2t} du\\
& = \left ( \frac{1}{1-2t} \right )^{r/2} \int_0^{\infty} \frac{u^{r/2-1} e^{-u/2} }{\Gamma(r/2) 2^{r/2}} du\\
& = \left ( \frac{1}{1-2t} \right )^{r/2}
\end{align*}
where the final integral is again Chi-Square and therefore has value 1.
Corollary 9.5.4. Chi-Square Properties via Moment Generating Function.
For the Chi-Square variable X,
\begin{equation*}
M(0) = 1
\end{equation*}
\begin{equation*}
M'(0) = r = \mu
\end{equation*}
\begin{equation*}
M''(0) = 2r + r^2 = \sigma^2 + \mu^2
\end{equation*}
Proof.
\begin{equation*}
M(0) = \left ( \frac{1}{1-2 \cdot 0} \right )^{r/2} = 1.
\end{equation*}
Continuing,
\begin{equation*}
M'(t) = \frac{r {\left(-2 t + 1\right)}^{\frac{1}{2} r - 1}}{{\left(-2 t + 1 \right)}^{r}}
\end{equation*}
and therefore
\begin{equation*}
M'(0) = \frac{r {\left(-2 \cdot 0 + 1\right)}^{\frac{1}{2} r - 1}}{{\left(-2 \cdot 0 + 1 \right)}^{r}} = \frac{r}{1} = r.
\end{equation*}
Continuing with the second derivative,
\begin{equation*}
M''(t) = -\frac{{\left(r - 2\right)} r {\left(-2 t + 1\right)}^{\frac{1}{2} r - 2}}{{\left(-2 t + 1\right)}^{r}} + \frac{2 r^{2} {\left(-2 t + 1\right)}^{r - 2}}{{\left(-2 t + 1\right)}^{\frac{3}{2} r}}
\end{equation*}
and therefore
\begin{align*}
M''(0) & = -\frac{{\left(r - 2\right)} r {\left(-2 \cdot 0 + 1\right)}^{\frac{1}{2} r - 2}}{{\left(-2 \cdot 0 + 1\right)}^{r}} + \frac{2 r^{2} {\left(-2 \cdot 0 + 1\right)}^{r - 2}}{{\left(-2 \cdot 0 + 1\right)}^{\frac{3}{2} r}} \\
& = -\frac{{\left(r - 2\right)} r }{1} + \frac{2 r^{2} }{1}= 2r + r^2 = \sigma^2 + \mu^2
\end{align*}
which is the squared mean plus the variance for the normal distribution.
It is interesting to note that the moment generating functions are not defined for the Cauchy Distribution 9.4.1 or for the Student’s t distribution 9.4.3.
