Administrative info
PA1 solutions on Piazza
HW4 out today, due Monday
PA2 out today
MT1 next Tuesday
Review session Sunday 7/10 5pm in 310 Soda
Review
Last time, we reviewed polynomials and saw how to reconstruct a
polynomial of degree d given d+1 points. Then we saw a procedure for
sharing a secret among n people such that k of them are needed to
recover it: construct a degree k-1 polynomial P(x) modulo some large
prime q such that P(0) ≡ s (mod q), the secret. Then give each
person a different point from the polynomial. If k of them
cooperate, they can reconstruct the degree k-1 P(x) and compute P(0)
to retrieve the secret.
Error Correction
We know turn our attention to another application of polynomials,
that of sending messages over unreliable networks.
Consider the following situations:
(1) FM radio has a range of ~10 miles, transmission power ~100 KW
(2) GPS satellite: 10,000 miles, 50 W
(reception power goes down by square of distance, so 2 billion
times weaker)
(3) Voyager space probe: 10 billion miles, 12-18 W
(10^22 times weaker)
Why is it that FM reception is flaky, but GPS works quite well, and
we can communicate with Voyager? Error correcting codes!
We want to be able to send long messages over unreliable channels
but still be able to recover the message. (For example, sending a
picture from a Neptune probe to Earth.) If a message is split into
multiple packets, some of those may be lost. Our assumption is that
the packets are labelled, so we know which packets are lost. We also
assume that packets either arrive correctly in their entirety or are
completely lost (i.e. they have been "erased"). (Maybe they arrive
corrupted, but we assume we can detect this and throw them away.)
Suppose we divide a message into n 100 bit packets, m_1, m_2, ...,
m_n. How can we send them so that they can be recovered reliably?
Let's assume that no more than k losses. (In practice, we'd pick
some k such that there is a very high probability that no more than
k are lost.)
We could just send each packet m_i many times, so that we can ensure
that at least one copy of each packet arrives at the destination.
But this is inefficient, since it transmits a lot of redundant
information.
Instead, we use polynomials to reduce the amount of redundant
information we have to send. We use the following scheme:
(1) Pick a prime q, q > 2^100 (so all 100 bit packets can be
represented). (Note: The receiver must know what q is! So
transmitter and receiver must agree beforehand on the value of
q.)
(2) Construct the degree n-1 polynomial over GF(q)
P(x) ≡ m_n x^{n-1} + m_{n-1} x^{n-1} + ... + m_2 x + m_1
(3) Transmit the n+k encoded packets P(1), P(2), ..., P(n+k).
Does this scheme work? We assumed that no more than k packets are
lost, so at least n packets arrive at Earth from Neptune. Then
mission control performs Lagrange interpolation to reconstruct P(x)
and reads off the resulting coefficients as the intended message.
Note that this scheme sends a lot less information than the naive
method of sending the same packet multiple times. It only requires
n+k packets rather than some c*n.
EX: Suppose we want to send the message (5, 0, 4, 1) with a maximum
of 2 packets getting lost. Then:
(1) We pick q = 7 > 5 so that all our packets can be
represented.
(2) We construct the degree 3 polynomial
P(x) ≡ 1x^3 + 4x^2 + 0x + 5 (mod 7)
≡ x^3 + 4x^2 + 5 (mod 7).
(3) We transmit n+k = 6 packets
P(1) ≡ 3 (mod 7)
P(2) ≡ 1 (mod 7)
P(3) ≡ 5 (mod 7)
P(4) ≡ 0 (mod 7)
P(5) ≡ 6 (mod 7)
P(6) ≡ 1 (mod 7)
Now as long as at least 4 packets, the receivier can reconstruct
P(x) ≡ 1x^3 + 4x^2 + 0x + 5 (mod 7)
and read off the message (5, 0, 4, 1).
This scheme is known as Reed-Solomon encoding. It was used by
Voyager and is used in DVDs, satellite communcitions, some digital
radio standards (e.g. DAB+), and in many other applications.
Compare this scheme with secret sharing. Let n be the number of
people who must be present out of a total of n+k people (where k is
the number of people who are "lost"). We wanted to "send" a single
packet, but we wanted to ensure that n packets arrived before the
"message" could be recovered. So we essentially padded the secret
with n-1 random packets so that it reached the required message size
of n. The secret is P(0) ≡ m_1 ≡ s, and m_2, ..., m_n are random
junk.
Finally, what happens if we no longer assume that we can detect
corrupted packets? We can still recover the message if we send n+2k
packets using the above encoding scheme, where at most k can be
corrupted or lost. The Berlekamp-Welch algorithm accomplishes this,
though we won't cover it in this class.
Summary
We have now completed the second unit of this course, modular
arithmetic. We studied the mathematical theory and then saw how to
use it in three very cool applications: encryption, secret sharing,
and error correction. We now turn our attention to the next unit in
the course, probability theory.
=======
Probability Theory
Probability theory is central to many aspects of EECS. For example,
we may want to determine how well an algorithm (e.g. quick sort)
works in the "average" case. Or algorithms may include randomness to
probabilistcally improve efficiency (e.g. hashing). Probability is
also used in artificial intelligence (infer most likely value of an
unknown quantity) and in many other areas of EECS.
Let's look at some simple examples of computing probabilities.
(1) If I flip a fair coin once, what is the chance of heads?
There are two possibilities, each of which are equally likely,
and only one of which is heads. So the probability should be
1/2.
(2) If I flip a fair coin twice, what is the chance of two heads?
There are now four possibilities: HH, TH, HT, and TT. (Note that
TH and HT are different, since one has heads in the first flip
and the other in the second.) Only one of these has two heads,
so the probability should be 1/4.
(3) If I flip a fair coin 100 times, what is the chance of exactly
50 heads?
It is no longer trivial. We need to count how many sequences of
100 flips have exactly 50 heads and the divided by the total
number of possible sequences. (The probability is ~8% or 0.08,
by the way.)
Thus, we need to learn how to count before we can move on to
computing probabilities.
Counting
We want to know how to count large quantities without actually
counting them. For example, how many seats are there in this room?
Instead of counting one-by-one, we count how many seats are in each
row and multiply by the number of rows. This is the basic idea:
count big quantities by counting smaller quantities and then
combining those to get a result for the big quantity. (Does this
remind you of anything? Recursion!)
Note that what we are doing in counting is counting the size of some
set S (e.g. the set of sequences of flips with exactly 50 heads, the
set of all possible sequences of flips). So if we can express S in
terms of smaller sets (not necessarily subsets), we can count the
sizes of the smaller sets to help us count the size of S.
We will now look at some general counting principles that can help
us in counting large quantities.
(1) Enumeration
Just list all possiblities. (We need to make sure that we've
covered everything!)
EX: How many bit strings of length 3 are there with no
consecutive zeroes?
ANS: 010, 011, 101, 110, 111
How do we make sure we covered every possiblity? One way is
to draw a decision tree:
Legal bit strings
0 010
/
0 - 1 - 1 011
/
/ 0 - 1 101
/ /
/ / 0 110
/ / /
x ---- 1 - 1 - 1 111
Bit # 1 2 3
EX: How many 2-out-of-3 game playoffs are there?
Winner sequence Winner
b bb b
/ / b bab b
b - a - a baa a
/ / b abb b
/ / b - a aba a
x -- a - a aa a