
Department of Electrical Engineering and Computer Science
Department of Mechanical Engineering

6.050J/2.110J – Information, Entropy and Computation –
Spring 2014


Unit 9: Maximum Entropy
Schedule
Lecture 
Friday, Apr 11, 2014, 1:00 PM 
Room 5234 
Recitation 
Wednesday, Apr 16, 2014, 1:00 PM 
Room 5234 
Recitation 
Friday, Apr 18, 2014, 1:00 PM 
Room 5234 
Lecture Handouts
Students who for any reason did not receive these items can pick them up in
Room 38344. Most of this material is also available on the 6.050J/2.110J
Web site
http://mtlsites.mit.edu/Courses/6.050.
Reading Assignment
Resources
Technical
The person most responsible for use of maximum entropy principles in various
fields of science is Edwin T. Jaynes (July 5, 1922–April 30, 1998).
The seminal papers are:

Jaynes, E. T., “Information Theory and Statistical Mechanics,”
Physical Review, vol. 106, no. 4, pp. 620630; May 15, 1957.
PDF (2.2
MB), PS (2.6
MB). This paper started the modern use of the Principle of Maximum Entropy
in physics

Jaynes, E. T., “Information Theory and Statistical Mechanics.
II,” Physical Review, vol. 108, no. 2, pp. 171190; October 15, 1957.
PDF (3.9
MB), PS (4.7
MB). Continuation of the previous reference
Historical
Books

The philosophy of assuming maximum uncertainty is discussed in Chapter 3 of
M. Tribus, “Thermostatics and Thermodynamics,” D. Van Nostrand
Co, Inc., Princeton, NJ; 1961

Another good explanation, in terms of estimating probabilities of an unfair
die is in E. T. Jaynes, “Information Theory and Statistical
Mechanics,” pp. 181218 in “Statistical Physics,” Brandeis
Summer Institute 1962, W. A. Benjamin, Inc., New York, NY; 1963.
PDF,
PS

Personal history by Jaynes, Edwin T. Jaynes, “Where Do We Stand on
Maximum Entropy?,” pp. 15118, in “The Maximum Entropy
Formalism,” Raphael D. Levine and Myron Tribus, editors, The MIT
Press, Cambridge, MA; 1979.
PDF,
PS
Help Wanted
6.050J/2.110J students: be the first to suggest a resource, for example a
useful Web site or a good book or article, to add to the list above. Send
your suggestion by email during Spring 2014 to 6.050staff at mit.edu.
6.050J/2.110J home page 
Spring 2014 
Search 
Comments and inquiries
Click here for information on MIT Accessibility