Section 10.3 Point Estimates
For Binomial, Geometric, what is p? For exponential, what is the mean? For normal, what are the mean and standard deviation? Each of these parameters are necessary before you can compute any probability values from their respective formulas. Since they might not be given in a particular instance, they will need to be estimated in some manner.
This estimate will have to be determined likely by utilizing sampling in some form. Since such an estimate will come from partial information (i.e. a sample) then it is very likely going to only be an approximation to the exact (but unknown) value. In general, an estimator is a numerical value which is used in the place of an unknown population statistic. To determine precisely what is a "best" estimator requires a multivariate approach and is beyond the scope of this text. Indeed, to justify why each of the following are good estimators look up the topic "Maximum Likelihood Estimators".
From your previous experience with the Poisson, Exponential, and Gamma distributions, you should also remember that each required a known value for \(\mu\) before proceeding with calculations. It is sensible to consider estimating the unknown population mean \(\mu\) using the sample mean
\begin{equation*}
\mu \approx \overline{x} = \frac{\sum x_k}{n}
\end{equation*}
where the values \(x_k\) are the n individual sample values.
For any continous variable and indeed for \(\overline{X}\text{,}\) \(P(\overline{X} = \mu) = 0\text{.}\) In general, you should expect a sample statistic to be close but not precisely equal to the population statistic. Indeed, if you were so lucky as to have the sample statistic to land on the population statistic, doing one more trial would mess things up anyway and the sample statistic would certainly change some.
In a similar manner with the Binomial, Geometric, and Negative Binomial distributions, you will remember that each required a known value for p before proceeding with any calculations. From our experiments we saw that relative frequency appeared to stabilize around what you might expect for the true proportion of success and therefore estimating the unknown proportion of success p using relative frequency
\begin{equation*}
p \approx \tilde{p} = \frac{y}{n}
\end{equation*}
where y is the number of successes in a collection of n bernoulli trials. Again, notice that the relative frequency \(\tilde{p}\) is technically an average as well so the probability that a given relative frequency will like exactly on the actual value of p is again zero.
Finally, the Normal distribution requires a numerical value for \(\sigma\text{,}\) the population’s standard deviation. It can be shown that the maximum likelihood estimator for \(\sigma^2\) is the variance v found in chapter one. However, you may remember that at that time we always adjusted this value somewhat using the formula \(s^2 = \frac{n}{n-1} v\) which increased the variance slightly. To uncover why you would not use the maximum likelihood estimator v requires you to look up the idea of "bias". As it turns out, v is maximum likelihood but exhibits mathematical bias whereas \(s^2\) is slightly suboptimal with respect to likelihood but exhibits no bias. Therefore, for estimating the unknown population variance \(\sigma^2\) you can use sample variance
\begin{equation*}
\sigma^2 \approx s^2
\end{equation*}
and similarly sample standard deviation
\begin{equation*}
\sigma \approx s
\end{equation*}
to approximate the theoretical standard deviation.
