Department of Electrical Engineering and Computer Science
 Department of Mechanical Engineering
 6.050J / 2.110J     Information and Entropy     Spring 2003


Unit 8: Maximum Entropy


Lecture Tuesday, Apr 8, 2003, 12:00 PM Room 37-212
Recitation Thursday, Apr 10, 2003, 12:00 PM Room 37-212
Problem Set Posted Friday, Apr 4, 2003 Due Friday, Apr 11, 2003
Solutions Posted Friday, Apr 11, 2003  
Quiz Thursday, Apr 24, 2003, 12:00 PM Room 37-212
Closed book except that one sheet of 8 1/2 x 11 inch
paper, with notes on both sides, is allowed.

Lecture Handouts

Students who for any reason did not receive these items can pick up a copy in Room 38-344. Most of this material is also available on the 6.050J/2.110J Web site

Reading Assignment



The person most responsible for use of maximum entropy principles in various fields of science is Edwin T. Jaynes. The seminal papers are:

  • Jaynes, E. T., "Information Theory and Statistical Mechanics," Physical Review, vol. 106, no. 4, pp. 620-630; May 15, 1957. PDF (2.2 MB), PS (2.6 MB). This paper started the use of the Principle of Maximum Entropy in physics
  • Jaynes, E. T., "Information Theory and Statistical Mechanics. II," Physical Review, vol. 108, no. 2, pp. 171-190; October 15, 1957. PDF (2.2 MB), PS (2.6 MB). Continuation of the previous reference



  • The philosophy of assuming maximum uncertainty is discussed in Chapter 3 of M. Tribus, "Thermostatics and Thermodynamics," D. Van Nostrand Co, Inc., Princeton, NJ; 1961
  • Another good explanation, in terms of estimating probabilities of an unfair die is in E. T. Jaynes, "Information Theory and Statistical Mechanics," pp. 181-218 in "Statistical Physics," Brandeis Summer Institute 1962, W. A. Benjamin, Inc., New York, NY; 1963. PDF, PS
  • Personal history by Jaynes, Edwin T. Jaynes, "Where Do We Stand on Maximum Entropy?," pp. 15-118, in "The Maximum Entropy Formalism," Raphael D. Levine and Myron Tribus, editors, The MIT Press, Cambridge, MA; 1979. PDF, PS

Help Wanted

6.050J/2.110J students: be the first to suggest a resource, for example a useful Web site or a good book or article, to add to the list above. If your suggestion is accepted by the 6.050J/2.110J staff, you will get a $5 ice-cream gift certificate. Send your suggestion by e-mail during Spring 2003 to 6.050-staff at

6.050J/2.110J home page | Spring 2003 | Site map | Search | About this page | Comments and inquiries Click here for information on MIT Accessibility