Department of Electrical Engineering and Computer Science
 Department of Mechanical Engineering
 6.050J / 2.110J     Information and Entropy


These notes are from the Spring 2004 offering. Notes from Spring 2003 are available here. Notes for the Spring 2005 offering will be here when available.

We would appreciate learning of any errors in the notes, and would be pleased to receive suggestions for improvements. Please use the comment form linked below.

Note that these notes are still under development. In particular, Chapter 13 is in very sketchy form, and many useful, illuminating examples are waiting to be inserted in other chapters.

Table of Contents

Preface (version 1.1.0)

Chapter 1. Bits (version 1.1.0)

Chapter 2. Codes (version 1.1.0)
  Transcribing RNA to Proteins

Chapter 3. Compression (version 1.1.0)

Chapter 4. Errors (version 1.1.0)

Chapter 5. Probability (version 1.1.0)

Chapter 6. Communications (version 1.1.0)

Chapter 7. Processes (version 1.1.0)

Chapter 8. Inference (version 1.1.0)

Chapter 9. Principle of Maximum Entropy (version 1.1.0)

Chapter 10. Physical Systems (version 1.1.0)

Chapter 11. Energy (version 1.1.0)

Chapter 12. Temperature (version 1.1.0)

Chapter 13. Quantum Information (version 1.1.0)

These notes are copyright © 2003, 2004 Massachusetts Institute of Technology

6.050J/2.110J home page | Spring 2004 | Site map | Search | About this page | Comments and inquiries Click here for information on MIT Accessibility