Department of Electrical Engineering and Computer Science Department of Mechanical Engineering 6.050J/2.110J – Information, Entropy and Computation – Spring 2010

Unit 9: Maximum Entropy

Schedule

Lecture Handouts

Students who for any reason did not receive these items can pick them up in Room 38-344. Most of this material is also available on the 6.050J/2.110J Web site http://mtlsites.mit.edu/Courses/6.050.

• Unit 9 Resources (this page)
• 6.050J/2.110J Notes,
  • Chapter 9, Principle of Maximum Entropy

Reading Assignment

• Notes, Chapter 9, Principle of Maximum Entropy

Resources

Technical

The person most responsible for use of maximum entropy principles in various fields of science is Edwin T. Jaynes (July 5, 1922 – April 30, 1998). The seminal papers are:

• Jaynes, E. T., “Information Theory and Statistical Mechanics,” Physical Review, vol. 106, no. 4, pp. 620-630; May 15, 1957. PDF (2.2 MB), PS (2.6 MB). This paper started the modern use of the Principle of Maximum Entropy in physics
• Jaynes, E. T., “Information Theory and Statistical Mechanics. II,” Physical Review, vol. 108, no. 2, pp. 171-190; October 15, 1957. PDF (3.9 MB), PS (4.7 MB). Continuation of the previous reference

Historical

• Joseph-Louis Lagrange (1736–1813) biography
• Edwin T. Jaynes (1922–1998) biography, photograph, bibliography
  
  

Books

• The philosophy of assuming maximum uncertainty is discussed in Chapter 3 of M. Tribus, “Thermostatics and Thermodynamics,” D. Van Nostrand Co, Inc., Princeton, NJ; 1961
• Another good explanation, in terms of estimating probabilities of an unfair die is in E. T. Jaynes, “Information Theory and Statistical Mechanics,” pp. 181-218 in “Statistical Physics,” Brandeis Summer Institute 1962, W. A. Benjamin, Inc., New York, NY; 1963. PDF, PS
• Personal history by Jaynes, Edwin T. Jaynes, “Where Do We Stand on Maximum Entropy?,” pp. 15-118, in “The Maximum Entropy Formalism,” Raphael D. Levine and Myron Tribus, editors, The MIT Press, Cambridge, MA; 1979. PDF, PS

Help Wanted

6.050J/2.110J students: be the first to suggest a resource, for example a useful Web site or a good book or article, to add to the list above. Send your suggestion by e-mail during Spring 2010 to 6.050-staff at mit.edu.