
Department of Electrical Engineering and Computer Science
Department of Mechanical Engineering

6.050J/2.110J – Information, Entropy and Computation –
Spring 2017


Unit 9: Maximum Entropy
Schedule
Lecture 
Wednesday, Apr 12, 2017, 7:30 PM 
Room 1136 
Quiz 
Wednesday, Apr 19, 2017, 7:30 PM 
Room 1136 
Closed book except that one sheet of 8 1/2 x 11 inch
paper, with notes on both sides, is allowed.
Coverage: through Unit 8

Lecture Handouts
Students who for any reason did not receive these items can pick them up in
Room 38344. Most of this material is also available on the 6.050J/2.110J
Web site
http://mtlsites.mit.edu/Courses/6.050.
Reading Assignment
Resources
Technical
The person most responsible for use of maximum entropy principles in various
fields of science is Edwin T. Jaynes (July 5, 1922–April 30, 1998).
The seminal papers are:

Jaynes, E. T., “Information Theory and Statistical Mechanics,”
Physical Review, vol. 106, no. 4, pp. 620630; May 15, 1957.
PDF (2.2
MB). This paper started the modern use of the Principle of Maximum Entropy
in physics

Jaynes, E. T., “Information Theory and Statistical Mechanics.
II,” Physical Review, vol. 108, no. 2, pp. 171190; October 15, 1957.
PDF (3.9
MB). Continuation of the previous reference
Historical
Books

The philosophy of assuming maximum uncertainty is discussed in Chapter 3 of
M. Tribus, “Thermostatics and Thermodynamics,” D. Van Nostrand
Co, Inc., Princeton, NJ; 1961

Another good explanation, in terms of estimating probabilities of an unfair
die is in E. T. Jaynes, “Information Theory and Statistical
Mechanics,” pp. 181218 in “Statistical Physics,” Brandeis
Summer Institute 1962, W. A. Benjamin, Inc., New York, NY; 1963.
PDF

Personal history by Jaynes, Edwin T. Jaynes, “Where Do We Stand on
Maximum Entropy?,” pp. 15118, in “The Maximum Entropy
Formalism,” Raphael D. Levine and Myron Tribus, editors, The MIT
Press, Cambridge, MA; 1979.
PDF
Help Wanted
6.050J/2.110J students: be the first to suggest a resource, for example a
useful Web site or a good book or article, to add to the list above. Send
your suggestion by email during Spring 2017 to 6.050staff at mit.edu.
6.050J/2.110J home page 
Spring 2017 
Search 
Comments and inquiries
Click here for information on MIT Accessibility