Definition of Probability

Section 4.3 Definition of Probability

Relative frequency gives a way to measure the proportion of "successful" outcomes when doing an experimental approach. From the interactive applications above, it appears that the relative frequency does jump around as the experiment is repeated but that the amount of variation decreases as the number of experiments increases. This is known to be true in general and is known as the "Law of Large Numbers".

🔗

We would like to formalize what these relative frequencies are approaching and will call this theoretical limit the "probability" of the outcome. In doing so, we will do our best to model our definition so that it follow the behavior of relative frequency.

🔗

To generate a general definition for probability, we need to know what is is that we measuring. In general, we will be finding the probability of sets of possible outcomes...that is, a subset of the Sample Space S. Toward that end, it is important to briefly look at some properties of sets.

🔗

Definition 4.3.1. Pairwise Disjoint Sets.

🔗

{A_{1}, A_{2}, . . ., A_{n}}

are pairwise disjoint provided

A_{k} \cap A_{j} = \emptyset

so long as

k \neq j .

Disjoint sets as also often called mutually exclusive.

🔗

Play around with the interactive cell below by adding and removing items in each of the three sets. Find elements so that the intersection of all three sets is empty but at least one of the paired sets are not disjoint. See if you can make all of the paired sets not disjoint but the intersection of all three disjoint. This is why we need to consider "pairwise" disjoint sets.


    
        
xxxxxxxxxx
 
1
def f(s, braces=True): 
2
    t = ', '.join(sorted(list(s)))
3
    if braces: return '{' + t + '}'
4
    return t
5
def g(s): return set(str(s).replace(',',' ').split())
6
7
@interact
8
def _(X='1,2,3', Y='2,a,3,4,apple', Z='a,b,10,apple'):
9
    S = [g(X), g(Y), g(Z)]
10
    X,Y,Z = S
11
    XY = X & Y
12
    XZ = X & Z
13
    YZ = Y & Z
14
    XYZ = XY & Z
15
16
    Txy = " - NOT disjoint "
17
    if Set(XY).is_empty():
18
        Txy = ' - disjoint '
19
    pretty_print(html("$X \\cap Y$ = %s"%f(XY)+"%s"%Txy))
20
    Txz = " - NOT disjoint "
21
    if Set(XZ).is_empty():
22
        Txz = ' - disjoint '
23
    pretty_print(html("$X \\cap Z$ = %s"%f(XZ)+"%s"%Txz))
24
    Tyz = " - NOT disjoint "
25
    if Set(YZ).is_empty():
26
        Tyz = ' - disjoint ' 
27
    pretty_print(html("$Y \\cap Z$ = %s"%f(YZ)+"%s"%Tyz))
28
    Txyz = " - NOT disjoint "
29
    if Set(XYZ).is_empty():
30
        Txyz = ' - disjoint ' 
31
    pretty_print(html("$X \\cap Y \\cap Z$ = %s"%f(XYZ)+"%s"%Txyz))
32
    centers = [(cos(n*2*pi/3), sin(n*2*pi/3)) for n in [0,1,2]]
33
    scale = 1.7
34
    clr = ['yellow', 'blue', 'green']
35
    G = Graphics()
36
    for i in range(len(S)):
37
        G += circle(centers[i], scale, rgbcolor=clr[i], 
38
             fill=True, alpha=0.3)
39
    for i in range(len(S)):
40
        G += circle(centers[i], scale, rgbcolor='black')
41
42
    # Plot what is in one but neither other
43
    for i in range(len(S)):
44
        Z = set(S[i])
45
        for j in range(1,len(S)):
46
            Z = Z.difference(S[(i+j)%3])
47
        G += text(f(Z,braces=False), (1.5*centers[i][0],1.7*centers[i][1]), 
48
                       rgbcolor='black')
49
50
51
    # Plot pairs of intersections
52
    for i in range(len(S)):
53
        Z = (set(S[i]) & S[(i+1)%3]) - set(XYZ)
54
        C = (1.3*cos(i*2*pi/3 + pi/3), 1.3*sin(i*2*pi/3 + pi/3))
55
        G += text(f(Z,braces=False), C, rgbcolor='black')
56
57
    # Plot intersection of all three
58
    G += text(f(XYZ,braces=False), (0,0), rgbcolor='black')
59
60
    # Show it
61
    G.show(aspect_ratio=1, axes=False)

    
    
    
    
        
            
                Language:
                
            
        
    
    




    
    
        
        Messages

🔗

Consider how we might create a definition for the expectation of a given outcome. To do so, first consider a desired collection of outcomes A. If each outcome in A is chosen randomly then we might consider using a formula similar to relative frequency and set a measure of expectation to be |A|/|S|. For example, on a standard 6-sided die, the expectation of the outcome A={2} from the collection S = {1,2,3,4,5,6} could be |A|/|S| = 1/6.

🔗

From our example where we take the sum of two die, the outcome A = { 4,5 } from the collection S = {2,3,4,...,12} would be

\begin{matrix} | A | = | {(1, 3), (2, 2), (3, 1), (1, 4), (2, 3), (3, 2), (4, 1)} | = 7 \\ | S | = | {(1, 1), . . ., (1, 6), (2, 1), . . ., (2, 6), . . ., (6, 1), . . ., (6, 6)} | = 36 \end{matrix}

🔗

and so the expected relative frequency would be |A|/|S| = 7/36. Compare this theoretical value with the sum of the two outcomes from your experiment above.

🔗

We are ready to now formally give a name to the theoretical measure of expectation for outcomes from an experiment. Taking our cue from our examples, let’s make our definition agree with the following relative frequency properties:

🔗
Relative frequency cannot be negative, since cardinality cannot be negative
🔗
Relative frequencies for complementary events should sum to one and for disjoint events these relative frequencies should simply accumulate.
🔗
Relative frequencies for collections of disjoint outcomes should equal the sum of the individual relative frequencies

🔗

which leads us to the following formal definition...

🔗

Definition 4.3.2. Probability.

🔗

The probability P(A) of a given outcome A is a set function that satisfies:

🔗

🔗
(Nonnegativity) P(A) $\geq 0$
🔗
(Totality) P(S) = 1
🔗
(Subadditivity) If A $\cap$ B = $\emptyset,$ then P(A $\cup$ B) = P(A) + P(B). In general, if { $A_{k}$ } are pairwise disjoint then $P (\cup_{k} A_{k}) = \sum_{k} P (A_{k}) .$

🔗

Checkpoint 4.3.3. WeBWorK - Using the definition.

Suppose you select a letter at random from the word MISSISSIPPI.

The probability of selecting the letter M is

The probability of selecting the letter S is

The probability of selecting the letters P or I is

The probability of not selecting the letter I is

Hint.

Count the number of letters in the word. When computing each probability, this is the number that goes on the bottom.

Answer 1.

\frac{1}{11}

Answer 2.

\frac{4}{11}

Answer 3.

\frac{2}{11} + \frac{4}{11}

Answer 4.

1 - \frac{4}{11}

Solution.

The probability of ’M’s is 1/11.

The probability of ’I’s is 4/11.

The probability of ’S’s is 4/11.

The probability of ’P’s is 2/11.

Notice when you are given complete information regarding the entire data set then determining probabilities for events can be relatively easy to compute.

🔗

Based upon this definition we can immediately establish a number of results.

🔗

Theorem 4.3.4. Probability of Complements.

🔗

For any event A,

P (A) + P (A^{c}) = 1

🔗

Proof.

Let A be any event and note that

A \cap A^{c} = \emptyset .

But

A \cup A^{c} = S .

So, by subadditivity 4.3.2

1 = P (S) = P (A \cup A^{c}) = P (A) + P (A^{c})

as desired.

🔗

Theorem 4.3.5.

🔗

P (\emptyset) = 0

🔗

Proof.

Note that

\emptyset^{c} = S .

So, by the theorem above 4.3.4,

1 = P (S) + P (\emptyset) \Rightarrow 1 = 1 + P (\emptyset) .

Cancelling the 1 on both sides gives

P (\emptyset) = 0 .

🔗

Theorem 4.3.6.

For events A and B

A \subseteq B \Rightarrow P (A) \leq P (B) .

🔗

Proof.

Assume sets A and B satisfy

A \subseteq B .

Then, notice that

A \cap (B - A) = \emptyset

and

B = A \cup (B - A) .

Therefore, by subadditivity and nonnegativity 4.3.2

\begin{matrix} 0 \leq P (B - A) \\ P (A) \leq P (A) + P (B - A) \\ P (A) \leq P (B) \end{matrix}

🔗

Theorem 4.3.7.

For any event A,

P (A) \leq 1

🔗

Proof.

Notice

A \subseteq S .

By the previous theorem 4.3.6

P (A) \leq P (S) = 1

🔗

Theorem 4.3.8.

For any sets A and B,

P (A \cup B) = P (A) + P (B) - P (A \cap B)

🔗

Proof.

Notice that we can write

A \cup B

as the disjoint union

A \cup B = (A - B) \cup (A \cap B) \cup (B - A) .

We can also write disjointly

\begin{matrix} A = (A - B) \cup (A \cap B) \\ B = (A \cap B) \cup (B - A) \end{matrix}

Hence,

\begin{aligned} P (A) & + P (B) - P (A \cap B) \\ = [P (A - B) + P (A \cap B)] \\ + [P (A \cap B) + P (B - A)] - P (A \cap B) \\ = P (A - B) + P (A \cap B) + P (B - A) \\ = P (A \cup B) \end{aligned}

🔗

This result can be extended to more that two sets using a property known as inclusion-exclusion. The following two theorems illustrate this property and are presented without proof.

🔗

Corollary 4.3.9.

🔗

For any sets A, B and C,

\begin{aligned} P (A \cup B \cup C) & = P (A) + P (B) + P (C) \\ - P (A \cap B) - P (A \cap C) - P (B \cap C) \\ + P (A \cap B \cap C) \end{aligned}

🔗

Corollary 4.3.10.

🔗

For any sets A, B, C and D,

\begin{aligned} P (A \cup B \cup C \cup D) & = P (A) + P (B) + P (C) + P (D) \\ - P (A \cap B) - P (A \cap C) - P (A \cap D) \\ - P (B \cap C) - P (B \cap D) - P (C \cap D) \\ + P (A \cap B \cap C) + P (A \cap B \cap D) \\ + P (A \cap C \cap D) + P (B \cap C \cap D) \\ - P (A \cap B \cap C \cap D) \end{aligned}

🔗

Many times, you will be dealing with making selections from a sample space where each item in the space has an equal chance of being selected. This may happen (for example) when items in the sample space are of equal size or when selecting a card from a completely shuffled deck or when coins are flipped or when a normal fair die is rolled.

🔗

It is important to notice that not all outcomes are equally likely--even in times when there are only two of them. Indeed, it is generally not an equally likely situation when picking the winner of a football game which pits, say, the New Orleans Saints professional football team with the New Orleans Home School Saints. Even though there are only two options the probability of the professional team winning in most years ought to be much greater than the chances that the high school will prevail.

🔗

When items are equally likely (sometimes also called "randomly selected") then each individual event has the same chance of being selected as any other. In this instance, determining the probability of a collection of outcomes is relatively simple.

🔗

Theorem 4.3.11. Probability of Equally Likely Events.

🔗

If outcomes in S are equally likely, then for

A \subseteq S,

P (A) = \frac{| A |}{| S |} .

🔗

Proof.

Enumerate S = {

x_{1}, x_{2}, . . ., x_{| S |}

} and note

P ({x_{k}}) = c

for some constant c since each item is equally likely. However, using each outcome as a disjoint event and the definition of probability,

\begin{aligned} 1 = P (S) & = P ({x_{1}} \cup {x_{2}} \cup . . . \cup {x_{| S |}}) \\ = P ({x_{1}}) + P ({x_{2}}) + . . . + P ({x_{| S |}}) \\ = c + c + . . . + c = | S | \times c \end{aligned}

and so

c = \frac{1}{| S |} .

Therefore,

P ({x_{k}}) = \frac{1}{| S |}

Hence, with A = {

a_{1}, a_{2}, . . ., a_{| A |}

}, breaking up the disjoint probabilities as above gives

\begin{aligned} P (A) & = P ({a_{1}} \cup {a_{2}} \cup . . . \cup {a_{| A |}}) \\ = P ({a_{1}}) + P ({a_{2}}) + . . . + P ({a_{| A |}}) \\ = \frac{1}{| S |} + \frac{1}{| S |} + . . . + \frac{1}{| S |} \\ = \frac{| A |}{| S |} \end{aligned}

as desired.


    
        
xxxxxxxxxx
 
1
var('A C D H J K Q S') 
2
3
def L(str):
4
    n = len(str)
5
    m = int(n/5)
6
    top = m+1
7
    if m == n/5:
8
        top = m
9
    for k in range(top):
10
        pretty_print(str[5*k:5*k+5])
11
        
12
suits = [S, D, C, H] 
13
values = [2, 3, 4, 5, 6, 7, 8, 9, 10, J, Q, K, A] 
14
15
deck = [(value, suit) for suit in suits for value in values]
16
full_deck = copy(deck)  # to save a copy
17
18
# L(deck)
19
shuffle(deck)
20
# L(deck)
21
deck1 = copy(full_deck)
22
shuffle(deck1)
23
24
@interact
25
def _(auto_update=False):
26
    global deck1    
27
    shuffle(deck1)            
28
    if (Set(deck1).cardinality() < 5):
29
     print('Deck is too small...getting a new deck')
30
        deck1 = copy(full_deck)
31
    else:
32
        hand = [deck1.pop() for card in range(5)] 
33
        pretty_print(html("The cards dealt:"))
34
        L(hand)
35
        pretty_print(html(" The remaining cards in the deck:"))
36
        L(deck1)
37
        pretty_print(html("\n The number of remaining cards in the deck "
38
                          +" = %s"%str(Set(deck1).cardinality())))

    
    
    
    
        
            
                Language:
                
            
        
    
    




    
    
        
        Messages

🔗

Let’s see if you understand the relationship between frequency and relative frequency. In this exercise, presume "Probabiity" to be the expected fraction of outcomes you might logically expect.

🔗

Checkpoint 4.3.12. WebWork - Equally Likely.

A fun size bag of M

a m p

Ms has about 15 candies. You open one of the bags and discover:

2 Blues, 2 Yellows, 5 Browns, 3 Reds and 3 Greens.

The probability of choosing a brown is .

The odds in favor of choosing a yellow is

The probability of choosing either a blue or a red is

The odds against a green being chosen is

Hint.

Odds in favor of an event = number of favorable outcomes / number of unfavorable outcomes.

Odds against an event = number of unfavorable outcomes / number of favorable outcomes.

Answer 1.

\frac{5}{15}

Answer 2.

\frac{2}{15 - 2}

Answer 3.

\frac{2 + 3}{15}

Answer 4.

\frac{15 - 3}{3}

So, by counting actual "equally likely" outcomes these probabilities are easy to compute.

🔗

Checkpoint 4.3.13. WebWork - Easy Probabilities.

(a)

Count the number of ways to arrange a sample of

2

elements from a population of

11

elements. NOTE: Order is not important.

answer:

(b)

If random sampling is to be employed, the probability that any particular sample will be selected is

Answer 1.

55

Answer 2.

0.0181818181818182

Notice how the probabilities look similar to relative frequencies. It’s just the case that you are counting ALL of the individual simple possibilities that lead to a success.

Essentials of Mathematical Probability and Statistics

Search Results:

Section 4.3 Definition of Probability

Definition 4.3.1. Pairwise Disjoint Sets.

Definition 4.3.2. Probability.

Checkpoint 4.3.3. WeBWorK - Using the definition.

Theorem 4.3.4. Probability of Complements.

Proof.

Theorem 4.3.5.

Proof.

Theorem 4.3.6.

Proof.

Theorem 4.3.7.

Proof.

Theorem 4.3.8.

Proof.

Corollary 4.3.9.

Corollary 4.3.10.

Theorem 4.3.11. Probability of Equally Likely Events.

Proof.

Checkpoint 4.3.12. WebWork - Equally Likely.

Checkpoint 4.3.13. WebWork - Easy Probabilities.