Unit 4: Probability

Probability is nothing but common sense reduced to calculation

-- Pierre-Simon Laplace (1749 - 1827)

Schedule

Lecture Handouts

Students who for any reason did not receive these items can pick up a copy in Room 38-344. Most of this material is also available on the 6.050J/2.110J Web site http://mtlsites.mit.edu/Courses/6.050.

• Unit 4 Resources (this page)
• Problem Set 4
• 6.050J/2.110J Notes
  • Chapter 5, Probability
• Towser's Wonderland Park greyhound handicaps, Boston Globe, March 4, 2002
• Part of Table of letter frequency in English (from "A Tale of Two Cities")
• David A. Huffmann, "A Method for the Construction of Minimum-Redundancy Codes," Proc. IRE, vol. 40, no. 9, pp. 1098-1101; September, 1952
• Page 13 of C. E. Shannon, "A Mathematical Theory of Communication"

Reading Assignment

• Notes, Chapter 5, Probability
• David A. Huffmann, "A Method for the Construction of Minimum-Redundancy Codes," Proc. IRE, vol. 40, no. 9, pp. 1098-1101; September, 1952

Resources

Technical

• David Salomon, "Data Compression," Springer; 1997. Huffman coding, Section 2.8; Facsimile Compression using Huffman coding, Section 2.13
• The Berkeley Mortality Database
• MIT student enrollment data: Y chart (all students) . . . Women students
• Probability tutorial, "Statistics for Engineers: Introduction to Probability Theory," from University of Wollongong, Australia

Historical

• F. N. David, "Games, Gods and Gambling," Charles Griffin and Co.; 1962 (Dover reprint 1998 in paperback)
• Girolamo Cardano (1501 - 1576), the first mathematician to calculate probabilities correctly
• Thomas Bayes (1702 - 1761)
• David A. Huffman (1925 - 1999) home page, left in place after his death; obituary

General Technical Books

There are many excellent texts on probability, many of which do not assume a familiarity with mathematics beyond introductory calculus. Most books on communications include a summary of the necessary background in probability.

• Alvin W. Drake, "Fundamentals of Applied Probability Theory," McGraw-Hill, Inc.; 1967; reprinted 1988. Prof. Drake taught 6.041 Probabilistic Systems Analysis for many years until he retired recently
• David Applebaum, "Probability and Information," Cambridge University Press; 1996. Chapter 4, Probability, contains a good comparison of the different philosophies underlying probability (symmetry, subjective, frequency)
• Simon Haykin, "Communication Systems," 4th edition, John Wiley and Sons, Inc.; 2001. Appendix 1, Probability Theory

There are many excellent texts on coding theory and communications, most of which assume a familiarity with mathematics beyond introductory calculus.

• John R. Pierce, "An Introduction to Information Theory: Symbols, Signals, and Noise," Dover Publications, Inc., New York, NY; 1961, 1980 (Second Edition). Mostly nonmathematical, by one of the nation's great scientific contributors at AT&T Bell Laboratories, who was also interested in reaching a general audience. He was later on the faculty at Caltech. One of his interesting sideline activities was writing science fiction stories under the pen name J. J. Coupling. He died April 2, 2002 at the age of 92
• Robert G. Gallager, "Information Theory and Reliable Communications," John Wiley & Sons, Inc., New York, NY; 1968. One of the early textbooks, designed for first-year graduate students, by one of the pioneers in communications, an MIT faculty member, later awarded the IEEE Medal of Honor, its most prestigious award
• Thomas M. Cover and Joy A. Thomas, "Elements of Information Theory," John Wiley & Sons, Inc., New York, NY; 1991. Aimed at university seniors and first-year graduate students. One of several excellent books of that era. Professor Cover, at Stanford University, is one of the leaders in Information Theory

Laboratory

• MIT EECS MATLAB Tutorial, used in 6.003; 1999
    http://mtlsites.mit.edu/Courses/6.050/2003/notes/matlab.pdf
  
• MATLAB on Athena, MIT; 1998
    http://web.mit.edu/olh/Matlab/Matlab.html
• Control Tutorials for Matlab, Carnegie Mellon University and the University of Michigan; August 11, 1997
    http://www.engin.umich.edu/group/ctm/basic/basic.html

Help Wanted

6.050J/2.110J students: be the first to suggest a resource, for example a useful Web site or a good book or article, to add to the list above. If your suggestion is accepted by the 6.050J/2.110J staff, you will get a $5 ice-cream gift certificate. Send your suggestion by e-mail during Spring 2003 to 6.050-staff at mit.edu.