Department of Electrical Engineering and Computer Science
Department of Mechanical Engineering
6.050J/2.110J – Information, Entropy and Computation – Spring 2008
Unit 9: Maximum Entropy
||Tuesday, Apr 8, 2008, 1:00 PM
||Thursday, Apr 10, 2008, 1:00 PM
||Tuesday, Apr 15, 2008, 1:00 PM
Closed book except that one sheet of 8 1/2 x 11 inch
paper, with notes on both sides, is allowed.
Coverage: through Unit 8
Students who for any reason did not receive these items can pick up a copy
in Room 38-344. Most of this material is also available on the
6.050J/2.110J Web site
The person most responsible for use of maximum entropy principles in various
fields of science is Edwin T. Jaynes. The seminal papers are:
Jaynes, E. T., "Information Theory and Statistical Mechanics," Physical
Review, vol. 106, no. 4, pp. 620-630; May 15, 1957.
MB), PS (2.6
MB). This paper started the use of the Principle of Maximum Entropy in
Jaynes, E. T., "Information Theory and Statistical Mechanics. II,"
Physical Review, vol. 108, no. 2, pp. 171-190; October 15, 1957.
MB), PS (4.7
MB). Continuation of the previous reference
The philosophy of assuming maximum uncertainty is discussed in Chapter 3 of
M. Tribus, "Thermostatics and Thermodynamics," D. Van Nostrand Co, Inc.,
Princeton, NJ; 1961
Another good explanation, in terms of estimating probabilities of an unfair
die is in E. T. Jaynes, "Information Theory and Statistical Mechanics," pp.
181-218 in "Statistical Physics," Brandeis Summer Institute 1962, W. A.
Benjamin, Inc., New York, NY; 1963.
Personal history by Jaynes, Edwin T. Jaynes, "Where Do We Stand on Maximum
Entropy?," pp. 15-118, in "The Maximum Entropy Formalism," Raphael D.
Levine and Myron Tribus, editors, The MIT Press, Cambridge, MA; 1979.
6.050J/2.110J students: be the first to suggest a resource, for example a
useful Web site or a good book or article, to add to the list above. Send
your suggestion by e-mail during Spring 2008
to 6.050-staff at mit.edu.
6.050J/2.110J home page |
Spring 2008 |
Site map |
this page |
Click here for information on MIT Accessibility