Information, Entropy and Intelligence
Develop the mathematical connections between thermodynamics, information theory, and Bayesian inference, and understand how they provide a toolkit for reasoning about the foundations of intelligent systems.
L172, Michaelmas 2026. Eight Tuesdays, 13 October – 1 December, in FW26. ACS MPhil / Part III.
Entropy appears in three traditions — thermodynamics, information theory, and Bayesian inference — and is the same mathematical object under different operational assumptions. The split we use throughout is: entropy forbids, probability prescribes. The module builds the machinery needed to treat superintelligence claims the way the second law treats perpetual motion, and it takes the human–machine bandwidth gap of The Atomic Human as a second, human, no-go.
Lectures
Eight two-hour lectures. In-class Moodle quizzes occupy the first ten minutes of lectures 2, 5, 7 and 8. Notes, slides, and notebooks are on the lectures page.
| Week | Date | Topic |
|---|---|---|
| 1 | 13 Oct | Introduction, Probability Review, and Motivation |
| 2 | 20 Oct | Boltzmann, Free Energy, and Entropy |
| 3 | 27 Oct | Shannon Entropy and the Partition Function |
| 4 | 3 Nov | Maxwell’s Demon and Landauer’s Principle |
| 5 | 10 Nov | Maximum Entropy and the Exponential Family |
| 6 | 17 Nov | Fisher Metric and Thermodynamic Length |
| 7 | 24 Nov | Projection, Natural Gradient, and Optimal Protocols |
| 8 | 1 Dec | Probability Transport and Limits on Intelligence |
Worksheets
Four take-home worksheets (notebook plus a short reflection). LLMs are permitted; each sheet is authenticated by the following in-class quiz. Briefs compile as LaMD practicals and are listed here.
Questions
The questions the module hopes to address, with the week a student should be able to define an answer and the week they should be able to interpret it in the course's voice, are on the questions page.
Assessment
Four worksheets at 15% each and four in-class quizzes at 10% each. Details and marking guidance are on the assessment page.
About
Prerequisites, learning outcomes, syllabus, and reading list: about the module.