Skip to main content

Section 5.3 Probability Functions

In the formulas below, we will presume that we have a random variable X which maps the sample space S onto some range of real numbers R. From this set, we then can define a probability function f(x) which acts on the numerical values in R and returns another real number. We attempt to do so to obtain (for discrete values) P(sample space value s)\(= f(X(s))\text{.}\) That is, the probability of a given outcome s is equal to the composition which takes s to a numerical value x which is then plugged into f to get the same final values.

For example, consider a random variable which assigns a 1 when you roll a 1 on a six-sided die and 0 otherwise. Presuming each side is equally likely,

Definition 5.3.1 Probability "Mass" Function
Given a discrete random variable X on a space R, a probability mass function on X is given by a function \(f:R \rightarrow \mathbb{R}\) such that:
\begin{align*} & \forall x \in R , f(x) \gt 0\\ & \sum_{x \in R} f(x) = 1\\ & A \subset R \Rightarrow P(X \in A) = \sum_{x \in A}f(x) \end{align*}
For \(x \not\in R\text{,}\) you can use the convention f(x)=0.
Definition 5.3.2 Probability "Density" Function
Given a continuous random variable X on a space R, a probability density function on X is given by a function \(f:R \rightarrow \mathbb{R}\) such that:
\begin{align*} & \forall x \in R , f(x) \gt 0\\ & \int_{R} f(x) dx = 1\\ & A \subset R \Rightarrow P(X \in A) = \int_{A} f(x) dx \end{align*}
For \(x \not\in R\text{,}\) you can use the convention f(x)=0.

For the purposes of this book, we will use the term "Probability Function" to refer to either of these options.

Consider \(f(x) = x/10\) over R = {1,2,3,4}. Then, f(x) is obviously positive for each of the values in R and certainly \(\sum_{x \in R} f(x) = f(1) + f(2) + f(3) + f(4) = 1/10 + 2/10 + 3/10 + 4/10 = 1\text{.}\) Therefore, f(x) is a probability mass function over the space R.

Consider \(f(x) = x^2/c\) for some positive real number c and presume R = [-1,2]. Then f(x) is nonnegative (and only equals zero at one point). To make f(x) a probability density function, we must have

\begin{equation*} \int_{x \in R} f(x) = 1. \end{equation*}

In this instance you get

\begin{equation*} 1 = \int_{-1}^2 x^2/c = x^3/(3c) |_{-1}^2 = \frac{8}{3c} - \frac{-1}{3c} = \frac{3}{c} \end{equation*}

Therefore, f(x) is a probability density function over R provided = 3.

Definition 5.3.5 Distribution Function
Given a random variable X on a space R, a probability distribution function on X is given by a function \(F:\mathbb{R} \rightarrow \mathbb{R}\) such that \(\displaystyle F(x)=P(X \le x)\)

Using \(f(x) = x/10\) over R = {1,2,3,4} again, note that F(x) will only change at these four domain values. We get

X F(x)
\(x \lt 1\) 0
\(1 \le x \lt 2\) 1/10
\(2 \le x \lt 3\) 3/10
\(3 \le x \lt 4\) 6/10
\(4 \le x \) 1
Table 5.3.7 Discrete Distribution Function Example

Consider \(f(x) = x^2/3\) over R = [-1,2]. Then, for \(-1 \le x \le 2\text{,}\)

\begin{equation*} F(x) = \int_{-1}^x u^2/3 du = x^3/9 + 1/9. \end{equation*}

Notice, F(-1) = 0 since nothing has yet been accumulated over values smaller than -1 and F(2)=1 since by that time everything has been accumulated. In summary:

X F(x)
\(x \lt -1\) 0
\(-1 \le x \lt 2\) \(x^3/9 + 1/9\)
\(2 \le x\) 1
Table 5.3.9 Continuous Distribution Function Example

Let a = inf(R). Then, for \(x \lt a,\)

\begin{equation*} F(x) = P(X \le x) \le P(X \lt a) = 0 \end{equation*}

since none of the x-values in this range are in R.

Let b = sup(R). Then, for

\begin{equation*} x \ge b, F(x) = P(X \le x) = P(X \le b) + P( b \lt X \le x) = P(X \le b) = 1 \end{equation*}

since all of the x-values in this range are in R and therefore will either sum over or integrate over all of R.

Case 1: R discrete

\begin{align*} \forall x_1,x_2 \in \mathbb{Z} \ni x_1 \lt x_2\\ F(x_2) & = \sum_{x \le x_2} f(x) \\ & = \sum_{x \le x_1} f(x) + \sum_{x_1 \lt x \le x_2} f(x)\\ & \ge \sum_{x \le x_1} f(x) = F(x_1) \end{align*}

Case 2: R continuous

\begin{align*} \forall x_1,x_2 \in \mathbb{R} \ni x_1 \lt x_2\\ F(x_2) & = \int_{-\infty}^{x_2} f(x) dx \\ & = \int_{-\infty}^{x_1} f(x) dx + \int_{x_1}^{x_2} f(x) dx\\ & \ge \int_{-\infty}^{x_1} f(x) dx\\ & = F(x_1) \end{align*}

Assume \(x \in R\) for some discrete R. Then,

\begin{equation*} F(x) - F(x-1) = \sum_{u \le x} f(u) - \sum_{u \lt x} f(u) = f(x) \end{equation*}

For a and b as noted, consider

\begin{align*} F(b) - F(a) & = \int_{-\infty}^b f(x) dx - \int_{-\infty}^a f(x) dx\\ & = \int_a^b f(x) dx \\ & = P(a \lt x \le b) \end{align*}

We will assume that F(x) is a continuous function. With that assumption, note

\begin{equation*} P(a-\epsilon \lt x \le a) = \int_{a-\epsilon}^a f(x) dx = F(a) - F(a-\epsilon) \end{equation*}

Take the limit as \(\epsilon \rightarrow 0^+\) to get the result noting that

Assume X is continuous and f and F as above. Notice, by the definition of f, \(\lim_{x \rightarrow \pm \infty} f(x) = 0\) since otherwise the integral over the entire space could not be finite.

Now, let A(x) be any antiderivative of f(x). Then, by the Fundamental Theorem of Calculus,

\begin{align*} F(x) & = \int_{-\infty}^x f(u) du\\ & = A(x) - \lim_{u \rightarrow -\infty} A(u) \end{align*}

Hence, \(F'(x) = A'(x) - \lim_{u \rightarrow -\infty} A'(u) = f(x)\) as desired.