6.050J/2.110J - Spring 2004 Unit 9

Department of Electrical Engineering and Computer Science
Department of Mechanical Engineering

6.050J / 2.110J Information and Entropy Spring 2004

Unit 9: Maximum Entropy

Schedule

Lecture	Monday, Apr 5, 2004, 1:00 PM	Room 2-105
Recitation	Wednesday, Apr 7, 2004, 1:00 PM	Room 2-105
Problem Set	Posted Friday, Apr 2, 2004	Due Friday, Apr 9, 2004
Solutions	Posted Friday, Apr 9, 2004
Quiz	Monday, Apr 12, 2004, 1:00 PM	Room 2-105
Quiz	Closed book except that one sheet of 8 1/2 x 11 inch paper, with notes on both sides, is allowed.

Lecture Handouts

Students who for any reason did not receive these items can pick up a copy in Room 38-344. Most of this material is also available on the 6.050J/2.110J Web sitehttp://mtlsites.mit.edu/Courses/6.050.

Unit 9 Resources (this page)
Problem Set 9
6.050J / 2.110J Notes
- Principle of Maximum Entropy
Professor Wyatt's Recitation Notes regarding Lagrange Multipliers

Reading Assignment

Principle of Maximum Entropy

Resources

Technical

The person most responsible for use of maximum entropy principles in various fields of science is Edwin T. Jaynes. The seminal papers are:

Jaynes, E. T., "Information Theory and Statistical Mechanics," Physical Review, vol. 106, no. 4, pp. 620-630; May 15, 1957. PDF (2.2 MB), PS (2.6 MB). This paper started the use of the Principle of Maximum Entropy in physics
Jaynes, E. T., "Information Theory and Statistical Mechanics. II," Physical Review, vol. 108, no. 2, pp. 171-190; October 15, 1957. PDF (2.2 MB), PS (2.6 MB). Continuation of the previous reference

Historical

Joseph-Louis Lagrange (1736 - 1813) biography
Adrien-Marie Legendre (1752 - 1833) biography
Edwin T. Jaynes (1922 - 1998) biography, photograph, bibliography

Books

The philosophy of assuming maximum uncertainty is discussed in Chapter 3 of M. Tribus, "Thermostatics and Thermodynamics," D. Van Nostrand Co, Inc., Princeton, NJ; 1961
Another good explanation, in terms of estimating probabilities of an unfair die is in E. T. Jaynes, "Information Theory and Statistical Mechanics," pp. 181-218 in "Statistical Physics," Brandeis Summer Institute 1962, W. A. Benjamin, Inc., New York, NY; 1963. PDF, PS
Personal history by Jaynes, Edwin T. Jaynes, "Where Do We Stand on Maximum Entropy?," pp. 15-118, in "The Maximum Entropy Formalism," Raphael D. Levine and Myron Tribus, editors, The MIT Press, Cambridge, MA; 1979. PDF, PS

Help Wanted

6.050J/2.110J students: be the first to suggest a resource, for example a useful Web site or a good book or article, to add to the list above. If your suggestion is accepted by the 6.050J/2.110J staff, you will get a $5 ice-cream gift certificate. Send your suggestion by e-mail during Spring 2004 to 6.050-staff (at) mit.edu.