Department of Electrical Engineering and Computer Science  Department of Mechanical Engineering  6.050J / 2.110J     Information and Entropy     Spring 2007

Unit 5: Probability

Probability is nothing but common sense reduced to calculation

-- Pierre-Simon Laplace (1749 - 1827)

Schedule

Lecture Handouts

Students who for any reason did not receive these items can pick up a copy in Room 38-344. Most of this material is also available on the 6.050J/2.110J Web site http://mtlsites.mit.edu/Courses/6.050.

• Unit 5 Resources (this page)
• 6.050J/2.110J Notes
  • Chapter 5, Probability
• David A. Huffmann, "A Method for the Construction of Minimum-Redundancy Codes," Proc. IRE, vol. 40, no. 9, pp. 1098-1101; September, 1952
• Page 13 of C. E. Shannon, "A Mathematical Theory of Communication"
• English Letter Usage Statistics (from "A Tale of Two Cities" by Charles Dickens)
• Towser's Wonderland Park greyhound handicaps, Boston Globe, February 27, 2005
      (and results, Boston Globe, February 28, 2005)

Reading Assignment

• Notes, Chapter 5, Probability
• David A. Huffmann, "A Method for the Construction of Minimum-Redundancy Codes," Proc. IRE, vol. 40, no. 9, pp. 1098-1101; September, 1952

Resources

Technical

• David Salomon, "Data Compression," Springer; 1997. Huffman coding, Section 2.8; Facsimile Compression using Huffman coding, Section 2.13
• The Human Mortality Database from University of California, Berkeley
• MIT current year student enrollment data: Y chart (all students) . . . Women students
• A Tutorial on Probability Theory, Paola Sebastiani, University of Massachusetts, Amherst. One of many good tutorials on the subject.

Historical

• F. N. David, "Games, Gods and Gambling," Charles Griffin and Co.; 1962 (Dover reprint 1998 in paperback)
• Girolamo Cardano (1501 - 1576), the first mathematician to calculate probabilities correctly
• Thomas Bayes (1702 - 1761)
• David A. Huffman (1925 - 1999) obituary

General Technical Books

There are many excellent texts on probability, many of which do not assume a familiarity with mathematics beyond introductory calculus. Most books on communications include a summary of the necessary background in probability.

• Alvin W. Drake, "Fundamentals of Applied Probability Theory," McGraw-Hill, Inc.; 1967; reprinted 1988. Prof. Drake taught 6.041 Probabilistic Systems Analysis for many years (he retired recently)
• Dimitri P. Bertsekas and John N. Tsitsiklis, "Introduction to Probability," Thena Scientific, Belmont, MA; 2002. Used in 6.041 today.
• David Applebaum, "Probability and Information," Cambridge University Press; 1996. Chapter 4, Probability, contains a good comparison of the different philosophies underlying probability (symmetry, subjective, frequency)
• Simon Haykin, "Communication Systems," 4th edition, John Wiley and Sons, Inc.; 2001. Appendix 1, Probability Theory

Help Wanted

6.050J/2.110J students: be the first to suggest a resource, for example a useful Web site or a good book or article, to add to the list above. If your suggestion is accepted by the 6.050J/2.110J staff, you will get a $5 ice-cream gift certificate. Send your suggestion by e-mail during Spring 2007 to 6.050-staff (at) mit.edu.