Develop the mathematical connections between thermodynamics, information theory, and Bayesian inference, and understand how they provide a toolkit for reasoning about the foundations of intelligent systems.
L172, Michaelmas 2026. Eight Tuesdays, 13 October – 1 December, in FW26. ACS MPhil / Part III.
Entropy appears in three traditions — thermodynamics, information theory, and Bayesian inference — and is the same mathematical object under different operational assumptions. The split we use throughout is: entropy forbids, probability prescribes. The module builds the machinery needed to treat superintelligence claims the way the second law treats perpetual motion, and it takes the human–machine bandwidth gap of The Atomic Human as a second, human, no-go.
Lectures
Eight two-hour lectures. In-class quizzes occupy the first ten minutes of lectures 2, 4, 6 and 8. Notes, slides, and notebooks are on the lectures page.
| Week | Date | Topic |
|---|---|---|
| 1 | 13 Oct | Introduction, Probability Review, and Motivation |
| 2 | 20 Oct | Ensemble, Entropy, and Embodiment |
| 3 | 27 Oct | Free Energy and Shannon Entropy |
| 4 | 3 Nov | Grouping and Maxwell’s Demon |
| 5 | 10 Nov | Maximum Entropy and the Exponential Family |
| 6 | 17 Nov | Exponential Family, Fisher, and Overviews |
| 7 | 24 Nov | Good Regulator, Goals, and Channels |
Worksheets
Four take-home worksheets. Worksheets 1–3 are Socratic LLM dialogues plus a handwritten reflection; worksheet 4 is a short notebook plus written reflection. LLMs are permitted on worksheets; each sheet is authenticated by the following in-class quiz. Briefs compile as LaMD practicals and are listed here.
Questions
The questions the module hopes to address, with the week a student should be able to define an answer and the week they should be able to interpret it in the course's voice, are on the questions page.
Assessment
Four worksheets at 15% each and four in-class quizzes at 10% each. Details and marking guidance are on the assessment page.
Getting help
Ask module questions on GitHub Discussions. Start from the pinned Course Q&A — start here post, then open a new Q&A discussion for each question. Keep personal or confidential matters (extensions, marks, candidate numbers) off Discussions — use email or the course office instead.
About
Prerequisites, learning outcomes, syllabus, and reading list: about the module.
Generative AI
How this course uses generative AI — lecturer practice and the student-facing rule — is on the generative AI page.