Department of Electrical Engineering and Computer Science
 Department of Mechanical Engineering
 6.050J / 2.110J     Information and Entropy     Spring 2003


Unit 7: Inference


Lecture Tuesday, Apr 1, 2003, 12:00 PM Room 37-212
Recitation Thursday, Apr 3, 2003, 12:00 PM Room 37-212
Problem Set Posted Friday, Mar 28, 2003 Due Friday, Apr 4, 2003
Solutions Posted Friday, Apr 4, 2003  
Quiz Thursday, Apr 24, 2003, 12:00 PM Room 37-212
Closed book except that one sheet of 8 1/2 x 11 inch
paper, with notes on both sides, is allowed.

Lecture Handouts

Students who for any reason did not receive these items can pick up a copy in Room 38-344. Most of this material is also available on the 6.050J/2.110J Web site

Reading Assignment



The person most responsible for use of maximum entropy principles in various fields of science is Edwin T. Jaynes (July 5, 1922 - April 30, 1998).

  • Jaynes, E. T., "Information Theory and Statistical Mechanics," Physical Review, vol. 106, no. 4, pp. 620-630; May 15, 1957. PDF (2.2 MB), PS (2.6 MB). This is the seminal paper which really started the modern use of the Principle of Maximum Entropy in physics
  • Jaynes, E. T., "Information Theory and Statistical Mechanics. II," Physical Review, vol. 108, no. 2, pp. 171-190; October 15, 1957. PDF (3.9 MB), PS (4.7 MB). Continuation of the previous reference


Jaynes knew, of course, about Thomas Bayes and when on sabbatical in England sought out and photographed the Bayes tombstone.


  • The philosophy of assuming maximum uncertainty is discussed in Chapter 3 of M. Tribus, "Thermostatics and Thermodynamics," D. Van Nostrand Co, Inc., Princeton, NJ; 1961
  • Another good explanation, in terms of estimating probabilities of an unfair die is in E. T. Jaynes, "Information Theory and Statistical Mechanics," pp. 181-218 in "Statistical Physics," Brandeis Summer Institute 1962, W. A. Benjamin, Inc., New York, NY; 1963. PDF, PS
  • Personal history by Jaynes, Edwin T. Jaynes, "Where Do We Stand on Maximum Entropy?," pp. 15-118, in "The Maximum Entropy Formalism," Raphael D. Levine and Myron Tribus, editors, The MIT Press, Cambridge, MA; 1979. PDF, PS

Help Wanted

6.050J/2.110J students: be the first to suggest a resource, for example a useful Web site or a good book or article, to add to the list above. If your suggestion is accepted by the 6.050J/2.110J staff, you will get a $5 ice-cream gift certificate. Send your suggestion by e-mail during Spring 2003 to 6.050-staff at

6.050J/2.110J home page | Spring 2003 | Site map | Search | About this page | Comments and inquiries Click here for information on MIT Accessibility