Administrative info
MT1 tomorrow! 5-7pm in 10 Evans
MT1 policies
- 1 cheat sheet (8.5x11, double sided)
- no calculators
HW4 solutions posted
HW5 posted
Review
Recall that a probability space consists of the following:
(1) A random experiment.
(2) The sample space (set of possible outcomes).
(3) The probability of each possible outcome of the experiment.
Further recall that the probabilities must satisfy the following:
(1) ∀ω∈Ω . 0 <= Pr[ω] <= 1
(2) ∑_{ω∈Ω} Pr[ω] = 1
An event is a subset of the sample space, i.e. a set of outcomes
from the sample space. The probability of an event E is the sum of
the probabilities of the outcomes in E:
Pr[E] = ∑_{ω∈E} Pr[ω].
In the case of a uniform distribution, this simplifies to
Pr[E] = |E|/|Ω|.
Probability Theory (cont.)
Let's start off with some examples to refresh our memory.
EX: If I toss a fair coin 100 times, what is the probability that
I get exactly 50 heads?
ANS: We know that this experiment has uniform distribution with
|Ω| = 2^100. Let E be the event that there are exactly
50 heads. Then we know from counting that |E| = C(100, 50).
Thus, Pr[E] = C(100, 50) or about 0.08.
EX: If I toss a fair coin 100 times, what is the probability that
I get more heads than tails?
ANS: Let F be the event that there are more heads than tails, G be
be the event that there are more tails than heads. Since
there is a 1-to-1 correspondence between outcomes in F and G,
|F| = |G|. Also note that the sets E (as defined above), F,
and G are disjoint, and that Ω = E ∪ F ∪ G.
Thus, |Ω| = |E| + |F| + |G| = |E| + 2|F| = C(100, 50) +
2|F| = 2^100. Thus, |F| = (2^100 - C(100, 50))/2, and Pr[F] =
|F|/|Ω| = 1/2 - C(100, 50)/2^101 ≈ 0.46.
EX: Suppose I roll a red and a blue die.
Ω = {(a, b) : 1 <= a,b <= 6}
Pr[ω] = 1/36 for all ω
What is the probability of
(a) the red die showing 6?
E = {(6, b) : 1 <= b <= 6}
Pr[E] = |E|/|Ω| = 1/6
(b) at least one die showing 6?
E1 = {(6, b) : 1 <= b <= 6}
E2 = {(a, 6) : 1 <= a <= 6}
E = E1 ∪ E2
|E| = |E1| + |E2| - |E1 ∩ E2| = 11
Pr[E] = 11/36
Note that in general, Pr[A ∪ B] = Pr[A] + Pr[B] - Pr[A
∩ B] whether or not the sample space has uniform
distribution.
(c) the dice sum to 7?
E = {(1, 6), (2, 5), (3, 4), (4, 3), (5, 2), (6, 1)}
Counting argument: for each choice of the value of the red
die i, there is exactly one choice for the blue die 7-i,
so there are 6 total choices.
Pr[E] = 1/6
(d) the dice sum to 10?
E = {(4, 6), (5, 5), (6, 4)}
Pr[E] = 1/12
EX: Suppose I roll two blue dice.
Ω = {(a, b) : 1 <= a <= b <= 6}
|Ω| = 21
Pr[(a, b)] = 1/36 if a = b, 1/18 if a != b. Note that this is
not a uniform distribution.
Let's make sure this sums to 1. There are 6 outcomes in which
a = b and 15 where a != b, so the sum is 6*1/36 + 15*1/18 =
1/6 + 5/6 = 1.
Now what is the probability of
(a) at least one die showing 6?
E = {(a, 6) : 1 <= a <= 6}
Here, Pr[E] != |E|/|Ω| = 6/21 = 2/7, since we don't
have a uniform distribution. So we have to do more work.
There is one outcome in E whose probability is 1/36 and 5
whose probability is 1/18. So we have
Pr[E] = 1/36 + 5*1/18 = 1/36 + 10/36 = 11/36.
Note that we got the same answer as in the distinguishable
dice case, as expected.
(b) the dice sum to 7?
E = {(1, 6), (2, 5), (3, 5)}
Each outcome has probability 1/18, so Pr[E] = 3/18 = 1/6.
As you can see, the choice of probability space affects the set of
outcomes and the probability distribution, as well as the difficulty
of the computations. However, the choice does not affect the end
results when we consider events of interest. The lesson here is to
always try to define the probability space to make probability
computations as simple as possible. In the two blue dice case, it
would have made things simpler if we had assumed that the two dice
were distinguishable and modeled the experiment accordingly.
Sometimes, however, we don't have the choice of a uniform
probability space.
EX: Suppose I toss a biased coin with 1/3 probability of heads.
Ω = {H, T}
Pr[H] = 1/3, Pr[T] = 2/3
Suppose I toss it three times?
Ω = {HHH, HHT, HTH, HTT, THH, THT, TTH, TTT}
Pr[ω] = ?
Intuitively, we'd expect that Pr[HHH] = (1/3)^3, Pr[HHT] =
(1/3)^2 * 2/3, and so on. Later on, we will see why this is the
case.
Birthday Paradox
There are 64 students enrolled in CS70. What is the likelihood that
some pair of students have the same birthday?
Let's model this as a random experiment that assigns birthdays to n
students, with equal probability for any outcome. We further assume
that there are exactly 365 days in a year. How many outcomes are
there? We have |Ω| = 365^n. Now in how many of these outcomes
do two people share the same birthday?
Let E be the event that two people share the same birthday. It's not
clear how to compute Pr[E] directly: we don't want to overcount the
cases in which three people have the same birthday, or four, and so
on.
Sometimes, it is easier to compute the probability of an event E's
complement Ω\E, which is the set of all outcomes that are not
in E. For simplicity, the complement of an event E is written as
E. In this case, E is the set of outcomes in which no
two students share the same birthday. What is Pr[E]? Well,
|E| = 365!/(365-n)! (by the product rule or n-permutation of
365 days), so Pr[E] = 365!/[(365-n)! 365^n].
Now how to get to Pr[E]? Well, E happens exactly when E does
not, so we expect Pr[E] + Pr[E] = 1. Thus, Pr[E] = 1 -
365!/[(365-n)! 365^n].
Plugging in values for n, we see that Pr[E] ≈ 0.51 when n =
23. So there is a greater than even chance that two people have the
same birthday out of a group of 23. For n = 64, we get Pr[E] ≈
0.997. So it's almost guaranteed that there are two CS70 students
who have the same birthday.
Monty Hall
Consider the following situation, based on a 1970s game show hosted
by Monty Hall. A contestant is shown three doors, one of which opens
to a prize and the remaining two of which open to goats. The
contestant must pick a door but does not open it. Hall's assistant
Carol opens one of the remaining doors, revealing a goat. The
contestant then can stick with his original door or switch to the
remaining unopened door. His final door choice is opened, and he
wins the prize only if it is behind his chosen door. What is the
contestant's best strategy?
Intuitively, it seems that since there are two remaining doors after
one with a goat has been opened, it makes no difference whether or
not the contestant switches. We will see that this is not actually
the case.
Let's model this problem. Define an outcome ω as a triple (i,
j, k), where i is the door where the prize is, j is the initial door
picked by the contestant, and k is the result of a coin flip by
Carol. If Carol has two doors to choose from (i.e. i = j), then she
opens the lower-numbered door if k = Heads and the higher-numbered
door if k = Tails. If she has no choice, then she ignores the coin
flip.
(Why did we add the coin flip? There is randomness in Carol's choice
of doors, and we need to make sure we model all the randomness in
the experiment. We keep the coin flip even if Carol has no choice in
order to preserve a uniform probability distribution.)
How many outcomes are there? There are 3 choices for each of i and j
and 2 for k, so there are 18 outcomes. It is reasonable to assume
that each outcome is equally likely, so we have a uniform
probability distribution where each outcome has probability 1/18.
Note that for each particular outcome ω, the contestant wins
for exactly one choice of strategy, staying or switching. If he
would lose in ω by staying, he would win by switching, and
vice versa.
Let's define A as the event that the contestant wins by staying with
his initial choice. What outcomes are in A? Clearly, it is the
sample points (i, j, k) for which i = j, so there are 6 outcomes in
A, and Pr[A] = 6/18 = 1/3. Then A is the event that he wins by
switching, so it must be that Pr[A] = 2/3. (We can also note
that A contains all the sample points (i, j, k) where i != j,
of which there are 12, so Pr[A] = 12/18 = 2/3.)
So the probability of winning by switching is 2/3, not 1/2!
Here's an intuitive way to think about this to see why the result
makes sense. Suppose without loss of generality that the contestant
picks door 1. Then Hall gives the contestant the option to switch
from door 1 to both of the doors 2 and 3. After he makes the choice
to stay or switch, Carol opens one of doors 2 and 3 to reveal a
goat. If he had switched, then he wins the prize if the other of
doors 2 and 3 has the prize. What do you think the probability of
winning is in this case? Can you see how this is the same as the
original problem? (In fact, the probability space is exactly the
same!)
This example illustrates why it is important to rigorously and
systematically compute probabilities. When this problem was
presented in Parade magazine in 1990, around 10,000 readers wrote in
claiming that the result was wrong, including almost 1000 with PhDs.
This demonstrates the danger of relying on intuition.