Department of Electrical Engineering and Computer Science  Department of Mechanical Engineering  6.050J / 2.110J     Information and Entropy     Spring 2005

Unit 5: Probability

Probability is nothing but common sense reduced to calculation

-- Pierre-Simon Laplace (1749 - 1827)

Schedule

Lecture Handouts

Students who for any reason did not receive these items can pick up a copy in Room 38-344. Most of this material is also available on the 6.050J/2.110J Web site http://mtlsites.mit.edu/Courses/6.050.

• Unit 5 Resources (this page)
• Problem Set 5
• 6.050J/2.110J Notes
  • Chapter 5, Probability
• David A. Huffmann, "A Method for the Construction of Minimum-Redundancy Codes," Proc. IRE, vol. 40, no. 9, pp. 1098-1101; September, 1952
• Page 13 of C. E. Shannon, "A Mathematical Theory of Communication"
• English Letter Usage Statistics (from "A Tale of Two Cities" by Charles Dickens)
• Towser's Wonderland Park greyhound handicaps, Boston Globe, February 27, 2005
      (and results, Boston Globe, February 28, 2005)

Reading Assignment

• Notes, Chapter 5, Probability
• David A. Huffmann, "A Method for the Construction of Minimum-Redundancy Codes," Proc. IRE, vol. 40, no. 9, pp. 1098-1101; September, 1952

Resources

Technical

• David Salomon, "Data Compression," Springer; 1997. Huffman coding, Section 2.8; Facsimile Compression using Huffman coding, Section 2.13
• The Human Mortality Database from University of California, Berkeley
• MIT student enrollment data: Y chart (all students) . . . Women students
• A Tutorial on Probability Theory, Paola Sebastiani, University of Massachusetts, Amherst. One of many good tutorials on the subject.

Historical

• F. N. David, "Games, Gods and Gambling," Charles Griffin and Co.; 1962 (Dover reprint 1998 in paperback)
• Girolamo Cardano (1501 - 1576), the first mathematician to calculate probabilities correctly
• Thomas Bayes (1702 - 1761)
• David A. Huffman (1925 - 1999) home page, left in place after his death; obituary

General Technical Books

There are many excellent texts on probability, many of which do not assume a familiarity with mathematics beyond introductory calculus. Most books on communications include a summary of the necessary background in probability.

• Alvin W. Drake, "Fundamentals of Applied Probability Theory," McGraw-Hill, Inc.; 1967; reprinted 1988. Prof. Drake taught 6.041 Probabilistic Systems Analysis for many years (he retired recently)
• Dimitri P. Bertsekas and John N. Tsitsiklis, "Introduction to Probability," Thena Scientific, Belmont, MA; 2002. Used in 6.041 today.
• David Applebaum, "Probability and Information," Cambridge University Press; 1996. Chapter 4, Probability, contains a good comparison of the different philosophies underlying probability (symmetry, subjective, frequency)
• Simon Haykin, "Communication Systems," 4th edition, John Wiley and Sons, Inc.; 2001. Appendix 1, Probability Theory

Laboratory

• Thomas F. Weiss, MIT, Introduction to Matlab, used in 6.003; Fall 1999
    http://mtlsites.mit.edu/Courses/6.050/2005/notes/matlab.pdf
  
• MATLAB on Athena, MIT; 1998
    http://web.mit.edu/olh/Matlab/Matlab.html
• Control Tutorials for Matlab, Carnegie Mellon University and the University of Michigan; August 18, 1997
  
  • Contents:  http://www.engin.umich.edu/group/ctm/
  • Matlab Basics Tutorial:  http://www.engin.umich.edu/group/ctm/basic/basic.html
• Kermit Sigmon, University of Florida, MATLAB Primer, second edition, translated to HTML by Nam Sun Wang, University of Maryland
    http://www.glue.umd.edu/~nsw/ench250/primer.htm

Help Wanted

6.050J/2.110J students: be the first to suggest a resource, for example a useful Web site or a good book or article, to add to the list above. If your suggestion is accepted by the 6.050J/2.110J staff, you will get a $5 ice-cream gift certificate. Send your suggestion by e-mail during Spring 2005 to 6.050-staff (at) mit.edu.