
Department of Electrical Engineering and Computer Science
Department of Mechanical Engineering

6.050J/2.110J – Information, Entropy and Computation –
Spring 2016


Unit 5: Probability
 Probability is nothing but common sense reduced to calculation
 —
PierreSimon
Laplace (1749–1827)
Schedule
Lecture 
Tuesday, Mar 1, 2016, 1:00 PM 
Room 1136 
Recitation 
Thursday, Mar 3, 2016, 1:00 PM 
Room 1136 
Lecture Handouts
Students who for any reason did not receive these items can pick them up in
Room 38344. Most of this material is also available on the 6.050J/2.110J
Web site
http://mtlsites.mit.edu/Courses/6.050.
 Unit 5 Resources (this page)

6.050J/2.110J Notes

David A. Huffman, “A Method for the Construction of MinimumRedundancy
Codes,” Proc. IRE, vol. 40, no. 9, pp. 10981101; September, 1952

English Letter Usage Statistics (from “A Tale of Two Cities” by
Charles Dickens, statistics compiled by Karl Hahn)

Towser's Wonderland Park greyhound handicaps, Boston Globe, February 27,
2005
(and results the next day, Boston Globe, February 28, 2005)
Reading Assignment

Notes, Chapter 5: Probability

David A. Huffman, “A Method for the Construction of MinimumRedundancy
Codes,” Proc. IRE, vol. 40, no. 9, pp. 10981101; September, 1952
Resources
Technical

David Salomon, “Data Compression,” Springer; 1997. Huffman
coding, Section 2.8; Facsimile Compression using Huffman coding, Section 2.13

The Human Mortality Database from
University of California, Berkeley

MIT current year student enrollment data:
Y chart
(all students) . . .
Women
students

A
Tutorial on Probability Theory, Paola Sebastiani, University of
Massachusetts, Amherst. One of many good tutorials on the subject
Historical

F. N. David, “Games, Gods and Gambling,” Charles Griffin and
Co.; 1962 (Dover reprint 1998 in paperback)

Girolamo
Cardano (1501–1576), the first mathematician to calculate
probabilities correctly

Thomas
Bayes (1702–1761)

David A.
Huffman (1925–1999)
General Technical Books
There are many excellent texts on probability, many of which do not assume a
familiarity with mathematics beyond introductory calculus. Most books on
communications include a summary of the necessary background in probability.

Alvin W. Drake, “Fundamentals of Applied Probability Theory,”
McGrawHill, Inc.; 1967; reprinted 1988. Prof. Drake taught 6.041
Probabilistic Systems Analysis for many years (he retired and then died Oct.
30, 2005.
Obituary.)

Dimitri P. Bertsekas and John N. Tsitsiklis, “Introduction to
Probability,” Thena Scientific, Belmont, MA; 2002. Used in 6.041
today.

David Applebaum, “Probability and Information,” Cambridge
University Press; 1996. Chapter 4, Probability, contains a good comparison
of the different philosophies underlying probability (symmetry, subjective,
frequency)

Simon Haykin, “Communication Systems,” 4th edition, John Wiley and Sons,
Inc.; 2001. Appendix 1, Probability Theory
Help Wanted
6.050J/2.110J students: be the first to suggest a resource, for example a
useful Web site or a good book or article, to add to the list above. Send
your suggestion by email during Spring 2016 to 6.050staff at mit.edu.
6.050J/2.110J home page 
Spring 2016 
Search 
Comments and inquiries
Click here for information on MIT Accessibility