Department of Electrical Engineering and Computer Science Department of Mechanical Engineering 6.050J/2.110J – Information, Entropy and Computation – Spring 2013

Unit 5: Probability

Probability is nothing but common sense reduced to calculation

— Pierre-Simon Laplace (1749–1827)

Schedule

Lecture Handouts

Students who for any reason did not receive these items can pick them up in Room 38-344. Most of this material is also available on the 6.050J/2.110J Web site http://mtlsites.mit.edu/Courses/6.050.

• Unit 5 Resources (this page)
• 6.050J/2.110J Notes
  • Chapter 5: Probability
• David A. Huffman, “A Method for the Construction of Minimum-Redundancy Codes,” Proc. IRE, vol. 40, no. 9, pp. 1098-1101; September, 1952
• English Letter Usage Statistics (from “A Tale of Two Cities” by Charles Dickens, statistics compiled by Karl Hahn)
• Towser's Wonderland Park greyhound handicaps, Boston Globe, February 27, 2005
      (and results the next day, Boston Globe, February 28, 2005)

Reading Assignment

• Notes, Chapter 5: Probability
• David A. Huffman, “A Method for the Construction of Minimum-Redundancy Codes,” Proc. IRE, vol. 40, no. 9, pp. 1098-1101; September, 1952

Resources

Technical

• David Salomon, “Data Compression,” Springer; 1997. Huffman coding, Section 2.8; Facsimile Compression using Huffman coding, Section 2.13
• The Human Mortality Database from University of California, Berkeley
• MIT current year student enrollment data: Y chart (all students) . . . Women students
• A Tutorial on Probability Theory, Paola Sebastiani, University of Massachusetts, Amherst. One of many good tutorials on the subject

Historical

• F. N. David, “Games, Gods and Gambling,” Charles Griffin and Co.; 1962 (Dover reprint 1998 in paperback)
• Girolamo Cardano (1501–1576), the first mathematician to calculate probabilities correctly
• Thomas Bayes (1702–1761)
• David A. Huffman (1925–1999)

General Technical Books

There are many excellent texts on probability, many of which do not assume a familiarity with mathematics beyond introductory calculus. Most books on communications include a summary of the necessary background in probability.

• Alvin W. Drake, “Fundamentals of Applied Probability Theory,” McGraw-Hill, Inc.; 1967; reprinted 1988. Prof. Drake taught 6.041 Probabilistic Systems Analysis for many years (he retired and then died Oct. 30, 2005. Obituary.)
• Dimitri P. Bertsekas and John N. Tsitsiklis, “Introduction to Probability,” Thena Scientific, Belmont, MA; 2002. Used in 6.041 today.
• David Applebaum, “Probability and Information,” Cambridge University Press; 1996. Chapter 4, Probability, contains a good comparison of the different philosophies underlying probability (symmetry, subjective, frequency)
• Simon Haykin, “Communication Systems,” 4th edition, John Wiley and Sons, Inc.; 2001. Appendix 1, Probability Theory

Help Wanted

6.050J/2.110J students: be the first to suggest a resource, for example a useful Web site or a good book or article, to add to the list above. Send your suggestion by e-mail during Spring 2013 to 6.050-staff at mit.edu.