Develop the mathematical connections between thermodynamics, information theory, and Bayesian inference, and understand how they provide a toolkit for reasoning about the foundations of intelligent systems.

L172, Michaelmas 2026. Eight Tuesdays, 13 October – 1 December, in FW26. ACS MPhil / Part III.

Entropy appears in three traditions — thermodynamics, information theory, and Bayesian inference — and is the same mathematical object under different operational assumptions. The split we use throughout is: entropy forbids, probability prescribes. The module builds the machinery needed to treat superintelligence claims the way the second law treats perpetual motion, and it takes the human–machine bandwidth gap of The Atomic Human as a second, human, no-go.

Lectures

Eight two-hour lectures. In-class quizzes occupy the first ten minutes of lectures 2, 4, 6 and 8. Notes, slides, and notebooks are on the lectures page.

WeekDateTopic
1 13 Oct Introduction, Probability Review, and Motivation
2 20 Oct Ensemble, Entropy, and Embodiment
3 27 Oct Free Energy and Shannon Entropy
4 3 Nov Grouping and Maxwell’s Demon
5 10 Nov Maximum Entropy and the Exponential Family
6 17 Nov Exponential Family, Fisher, and Overviews
7 24 Nov Good Regulator, Goals, and Channels

Worksheets

Four take-home worksheets. Worksheets 1–3 are Socratic LLM dialogues plus a handwritten reflection; worksheet 4 is a short notebook plus written reflection. LLMs are permitted on worksheets; each sheet is authenticated by the following in-class quiz. Briefs compile as LaMD practicals and are listed here.

Questions

The questions the module hopes to address, with the week a student should be able to define an answer and the week they should be able to interpret it in the course's voice, are on the questions page.

Assessment

Four worksheets at 15% each and four in-class quizzes at 10% each. Details and marking guidance are on the assessment page.

Getting help

Ask module questions on GitHub Discussions. Start from the pinned Course Q&A — start here post, then open a new Q&A discussion for each question. Keep personal or confidential matters (extensions, marks, candidate numbers) off Discussions — use email or the course office instead.

About

Prerequisites, learning outcomes, syllabus, and reading list: about the module.

Generative AI

How this course uses generative AI — lecturer practice and the student-facing rule — is on the generative AI page.