
Department of Electrical Engineering and Computer Science
Department of Mechanical Engineering

6.050J/2.110J – Information, Entropy and Computation – Spring 2008


Unit 5: Probability
 Probability is nothing but common sense reduced to calculation
 
PierreSimon
Laplace (1749  1827)
Schedule
Lecture 
Tuesday, Mar 4, 2008, 1:00 PM 
Room 36153 
Recitation 
Thursday, Mar 6, 2008, 1:00 PM 
Room 36153 
Lecture Handouts
Students who for any reason did not receive these items can pick up a copy
in Room 38344. Most of this material is also available on the
6.050J/2.110J Web site
http://mtlsites.mit.edu/Courses/6.050.

Unit 5
Resources (this page)

6.050 Notes,
Chapter 5

David A. Huffmann, "A Method for the Construction of MinimumRedundancy
Codes," Proc. IRE, vol. 40, no. 9, pp. 10981101; September, 1952

English
Letter Usage Statistics (from "A Tale of Two Cities" by Charles Dickens)

Towser's Wonderland Park greyhound handicaps, Boston Globe, February 27,
2005
(and results, Boston Globe, February 28, 2005)
Reading Assignment

Notes, Chapter 5, Probability

David A. Huffmann, "A Method for the Construction of MinimumRedundancy
Codes," Proc. IRE, vol. 40, no. 9, pp. 10981101; September, 1952
Resources
Technical

David Salomon, "Data Compression," Springer; 1997. Huffman coding, Section
2.8; Facsimile Compression using Huffman coding, Section 2.13

The Human Mortality Database from
University of California, Berkeley

MIT current year student enrollment data:
Y chart
(all students) . . .
Women
students

A
Tutorial on Probability Theory, Paola Sebastiani, University of
Massachusetts, Amherst. One of many good tutorials on the subject.
Historical

F. N. David, "Games, Gods and Gambling," Charles Griffin and Co.; 1962
(Dover reprint 1998 in paperback)

Girolamo
Cardano (1501  1576), the first mathematician to calculate
probabilities correctly

Thomas
Bayes (1702  1761)

David A. Huffman (1925  1999)
obituary
General Technical Books
There are many excellent texts on probability, many of which do not assume a
familiarity with mathematics beyond introductory calculus. Most books on
communications include a summary of the necessary background in probability.

Alvin W. Drake, "Fundamentals of Applied Probability Theory," McGrawHill,
Inc.; 1967; reprinted 1988. Prof. Drake taught 6.041 Probabilistic Systems
Analysis for many years (he retired and then died Oct. 30, 2005.
Obituary.)

Dimitri P. Bertsekas and John N. Tsitsiklis, "Introduction to
Probability," Thena Scientific, Belmont, MA; 2002. Used in 6.041 today.

David Applebaum, "Probability and Information," Cambridge University Press;
1996. Chapter 4, Probability, contains a good comparison of the different
philosophies underlying probability (symmetry, subjective, frequency)

Simon Haykin, "Communication Systems," 4th edition, John Wiley and Sons,
Inc.; 2001. Appendix 1, Probability Theory
Help Wanted
6.050J/2.110J students: be the first to suggest a resource, for example a
useful Web site or a good book or article, to add to the list above.
Send your suggestion by email during Spring 2008
to 6.050staff at mit.edu.
6.050J/2.110J home page 
Spring 2008 
Site map 
Search 
About
this page 
Comments
and inquiries
Click here for information on MIT Accessibility