
Unit 4: Probability
 Probability is nothing but common sense reduced to calculation
 
PierreSimon
Laplace (1749  1827)
Schedule
Lecture 
Tuesday, Mar 4, 2003, 12:00 PM 
Room 37212 
Recitation 
Thursday, Mar 6, 2003, 12:00 PM 
Room 37212 
Problem Set 
Posted Friday, Feb 28, 2003 
Due Friday, Mar 7, 2003 
Solutions 
Posted Friday, Mar 7, 2003 

Lecture Handouts
Students who for any reason did not receive these items can pick up a copy in
Room 38344. Most of this material is also available on the 6.050J/2.110J Web site
http://mtlsites.mit.edu/Courses/6.050.
 Unit 4 Resources (this page)
 Problem Set 4
 6.050J/2.110J Notes
 Towser's Wonderland Park greyhound handicaps, Boston Globe, March 4, 2002
 Part of Table
of letter frequency in English (from "A Tale of Two Cities")
 David A. Huffmann, "A Method for the Construction of MinimumRedundancy Codes,"
Proc. IRE, vol. 40, no. 9, pp. 10981101; September, 1952
 Page 13 of C. E. Shannon, "A Mathematical Theory of Communication"
Reading Assignment
 Notes, Chapter 5, Probability
 David A. Huffmann, "A Method for the Construction of MinimumRedundancy Codes,"
Proc. IRE, vol. 40, no. 9, pp. 10981101; September, 1952
Resources
Technical
 David Salomon, "Data Compression," Springer; 1997. Huffman coding, Section 2.8;
Facsimile Compression using Huffman coding, Section 2.13
 The Berkeley Mortality Database
 MIT student enrollment data:
Y chart (all students)
. . . Women students
 Probability tutorial,
"Statistics for Engineers: Introduction to Probability Theory," from University
of Wollongong, Australia
Historical
 F. N. David, "Games, Gods and Gambling," Charles Griffin and Co.; 1962 (Dover
reprint 1998 in paperback)
 Girolamo Cardano
(1501  1576), the first mathematician to calculate probabilities correctly
 Thomas Bayes
(1702  1761)
 David A. Huffman (1925  1999)
home page, left in place
after his death;
obituary
General Technical Books
There are many excellent texts on probability, many of which do not
assume a familiarity with mathematics beyond introductory calculus. Most books on
communications include a summary of the necessary background in probability.
 Alvin W. Drake, "Fundamentals of Applied Probability Theory," McGrawHill,
Inc.; 1967; reprinted 1988. Prof. Drake taught 6.041 Probabilistic Systems Analysis
for many years until he retired recently
 David Applebaum, "Probability and Information," Cambridge University Press;
1996. Chapter 4, Probability, contains a good comparison of the different philosophies
underlying probability (symmetry, subjective, frequency)
 Simon Haykin, "Communication Systems," 4th edition, John Wiley and Sons, Inc.;
2001. Appendix 1, Probability Theory
There are many excellent texts on coding theory and communications, most of which
assume a familiarity with mathematics beyond introductory calculus.
 John R. Pierce, "An Introduction to Information Theory: Symbols, Signals, and
Noise," Dover Publications, Inc., New York, NY; 1961, 1980 (Second Edition).
Mostly nonmathematical, by one of the nation's great scientific contributors at
AT&T Bell Laboratories, who was also interested in reaching a general audience.
He was later on the faculty at Caltech. One of his interesting sideline activities was writing
science fiction stories under the pen name J. J. Coupling. He died April 2, 2002
at the age of 92
 Robert G. Gallager, "Information Theory and Reliable Communications," John
Wiley & Sons, Inc., New York, NY; 1968. One of the early textbooks, designed
for firstyear graduate students, by one of the pioneers in communications, an MIT
faculty member, later awarded the IEEE Medal of Honor, its most prestigious award
 Thomas M. Cover and Joy A. Thomas, "Elements of Information Theory," John
Wiley & Sons, Inc., New York, NY; 1991. Aimed at university seniors and firstyear
graduate students. One of several excellent books of that era. Professor Cover,
at Stanford University, is one of the leaders in Information Theory
Laboratory
Help Wanted
6.050J/2.110J students: be the first to suggest a resource, for example a useful Web
site or a good book or article, to add to the list above. If your suggestion is
accepted by the 6.050J/2.110J staff, you will get a $5 icecream gift certificate.
Send your suggestion by email during Spring 2003 to 6.050staff at mit.edu.
