

Department of Electrical Engineering and Computer Science
Department of Mechanical Engineering 
6.050J / 2.110J Information and Entropy Spring 2005


Unit 5: Probability
 Probability is nothing but common sense reduced to calculation
 
PierreSimon
Laplace (1749  1827)
Schedule
Lecture 
Tuesday, Mar 1, 2005, 12:00 PM 
Room 2105 
Recitation 
Thursday, Mar 3, 2005, 12:00 PM 
Room 2105 
Problem Set 
Posted Monday, Feb 28, 2005 
Due Friday, Mar 4, 2005 
Solutions 
Posted Friday, Mar 4, 2005 

Lecture Handouts
Students who for any reason did not receive these items can pick up a copy in
Room 38344. Most of this material is also available on the 6.050J/2.110J Web site
http://mtlsites.mit.edu/Courses/6.050.
 Unit 5 Resources (this page)
 Problem Set 5
 6.050J/2.110J Notes
 David A. Huffmann, "A Method for the Construction of MinimumRedundancy Codes,"
Proc. IRE, vol. 40, no. 9, pp. 10981101; September, 1952
 Page 13 of C. E. Shannon, "A Mathematical Theory of Communication"
 English
Letter Usage Statistics (from "A Tale of Two Cities" by Charles Dickens)
 Towser's Wonderland Park greyhound handicaps, Boston Globe, February 27, 2005
(and results, Boston Globe, February 28, 2005)
Reading Assignment
 Notes, Chapter 5, Probability
 David A. Huffmann, "A Method for the Construction of MinimumRedundancy Codes,"
Proc. IRE, vol. 40, no. 9, pp. 10981101; September, 1952
Resources
Technical
 David Salomon, "Data Compression," Springer; 1997. Huffman coding, Section 2.8;
Facsimile Compression using Huffman coding, Section 2.13
 The Human Mortality Database from
University of California, Berkeley
 MIT student enrollment data:
Y chart (all students)
. . . Women students
 A Tutorial
on Probability Theory, Paola Sebastiani, University of Massachusetts, Amherst.
One of many good tutorials on the subject.
Historical
 F. N. David, "Games, Gods and Gambling," Charles Griffin and Co.; 1962 (Dover
reprint 1998 in paperback)
 Girolamo Cardano
(1501  1576), the first mathematician to calculate probabilities correctly
 Thomas Bayes
(1702  1761)
 David A. Huffman (1925  1999)
home page, left in place
after his death;
obituary
General Technical Books
There are many excellent texts on probability, many of which do not
assume a familiarity with mathematics beyond introductory calculus. Most books on
communications include a summary of the necessary background in probability.
 Alvin W. Drake, "Fundamentals of Applied Probability Theory," McGrawHill,
Inc.; 1967; reprinted 1988. Prof. Drake taught 6.041 Probabilistic Systems Analysis
for many years (he retired recently)
 Dimitri P. Bertsekas and John N. Tsitsiklis, "Introduction to Probability,"
Thena Scientific, Belmont, MA; 2002. Used in 6.041 today.
 David Applebaum, "Probability and Information," Cambridge University Press;
1996. Chapter 4, Probability, contains a good comparison of the different philosophies
underlying probability (symmetry, subjective, frequency)
 Simon Haykin, "Communication Systems," 4th edition, John Wiley and Sons, Inc.;
2001. Appendix 1, Probability Theory
Laboratory
 Thomas F. Weiss, MIT, Introduction to Matlab, used in 6.003; Fall 1999
http://mtlsites.mit.edu/Courses/6.050/2005/notes/matlab.pdf
 MATLAB on Athena, MIT; 1998
http://web.mit.edu/olh/Matlab/Matlab.html
 Control Tutorials for Matlab, Carnegie Mellon
University and the University of Michigan; August 18, 1997
 Kermit Sigmon, University of Florida, MATLAB Primer, second edition,
translated to HTML by Nam Sun Wang, University of Maryland
http://www.glue.umd.edu/~nsw/ench250/primer.htm
Help Wanted
6.050J/2.110J students: be the first to suggest a resource, for example a useful Web
site or a good book or article, to add to the list above. If your suggestion is
accepted by the 6.050J/2.110J staff, you will get a $5 icecream gift certificate.
Send your suggestion by email during Spring 2005
to 6.050staff (at) mit.edu.
6.050J/2.110J home page 
Spring 2005 
Site map 
Search 
About this page 
Comments and inquiries
Click here for information on MIT Accessibility