Maxwell’s Demon and Landauer’s Principle

Neil D. Lawrence

FW26, William Gates Building

Define This Week

  • What is Maxwell’s demon?
  • Why cannot feedback upgrade a car engine like ATP synthase?
  • Information and intelligence? (first cut)
  • Information constraints on a human? (bandwidth)

After This Lecture

  • Worksheet 2 released; due 3 November

This Session

  • Maxwell’s demon; membrane detailed balance
  • Information engines; car vs ATP synthase
  • Landauer; information and intelligence: first cut

Information, entropy and intelligence course notebook setup

Maxwell’s Demon

Maxwell’s demon seems to act against the second law Clausius had made explicit thirty years earlier.

  • Carnot (1824): engine efficiency has a ceiling
  • Clausius (1850s–1865): second law; entropy named and conserved in the books
  • Maxwell (1867): a demon seems to violate that law
  • Landauer (1961): erasure cost restores Clausius’s bookkeeping

Maxwell’s Demon

Velocity-bin entropy:

Detailed Balance Across the Membrane

The JavaScript demon is a selective membrane: one speed threshold \(v_\star\), opposite rules on each side.

  • Left \(\to\) right: pass only if \(v > v_\star\) (hot)
  • Right \(\to\) left: pass only if \(v \le v_\star\) (cold)
  • Elastic collisions remix velocities; the membrane only gates crossings

In a steady state the number of balls leaving left equals the number leaving right — detailed balance of membrane crossings.

  • Assume each side thermalises to a 2D Maxwellian at its own \(T_L\), \(T_R\)
  • Particle flux \(L\to R\): only the hot tail, \(v>v_\star\)
  • Particle flux \(R\to L\): only the cold body, \(v\le v_\star\)
  • Steady state: \(\Phi_L^{\mathrm{hot}}(n_L,T_L)=\Phi_R^{\mathrm{cold}}(n_R,T_R)\)

For the simulation budget, particle-flux balance at equal \(N\) yields \(T_L\approx 1.9\), \(T_R\approx 23\) (\(k_B=1\)) — a large split.

  • Unconstrained equilibrium: one \(T_0=\langle\mathrm{KE}\rangle\), entropy \(S_0\)
  • Membrane steady state: \(S_{\mathrm{ss}}<S_0\) (here \(\Delta S\approx -24\) nats)
  • The gas alone looks like a second-law violation

Particle balance and energy balance cannot both hold for two Maxwellians — the true steady state depletes velocity tails.

  • Hot crossings carry more energy per ball than cold crossings
  • Equal number fluxes still leave a net energy flux under Maxwellians
  • Collisions rebuild the tails; the membrane continually sculpts them
  • Local-\(T\) picture still predicts the direction and an entropy deficit

The membrane implements Maxwell’s sorting policy. The gas entropy falls; Clausius is not repealed.

  • Prescription: the velocity gate (who may cross)
  • No-go: you cannot harvest that \(\Delta S\) for free in a cycle
  • Next: Szilard makes the bit explicit; Landauer prices erasure

A Different Perspective

John Ellis (Maxwell Day 2024): the demon fails because measurement and a located trap door already dissipate.

  • Dissipation ledger: defined results (scatter, switch, barrier) already pay thermodynamically
  • Information ledger: Szilard \(\to\) stored outcome \(\to\) Landauer erasure \(\to\) Parrondo bound
  • Bennett (1982): erasure cost from phase-space compression — not circular (Bennett, 1982)
  • Both restore Clausius; they disagree on the bookkeeping variable
  • Worksheet 2 asks: which account matches the generality of the second law?

Szilard’s Engine

Szilard: knowing which half holds the molecule lets an isothermal expansion extract \(W_{\mathrm{ext}} = k_B T \ln 2\).

  • Shannon count: \(H(M)=\ln 2\) nats for unbiased left/right (\(1\) bit)
  • Thermodynamic count: \(W_{\mathrm{ext}} = k_B T H(M)\) in natural units
  • Same uncertainty; one names bits, the other names extractable work

Feynman treats a known microstate as fuel — a low-entropy tape you can spend.

  • Lectures on Computation (Ch. 5): Szilard step repeated; each stroke needs a fresh bit
  • Vol. I, Ch. 46: a ratchet alone cannot mine a single bath — fluctuations break the pawl
  • Chain: Maxwell \(\to\) Szilard \(\to\) measurement \(\to\) work \(\to\) erasure

Clausius’s second law says nothing about information; reconciling the pictures is a modern subject.

  • Task 1: refine the second law — feedback work \(W \ge -k_B T\, I(X;M)\)
  • Task 2: information is physical — outcomes live in metastable memory states
  • Cycle: measure \(\to\) feedback \(\to\) reset (Landauer on the memory)

Information Engines

An information engine converts stored or acquired information into extractable work — Szilard is the textbook case.

  • First model of intelligence (information-engines talk): policy under a thermodynamic ledger
  • Feedback bound (Parrondo / Sagawa–Ueda): \(W \ge -k_B T\, I(X;M)\) on average in a cycle
  • Memory is physical: metastable states; equal double well next
  • Prescription: measurement policy; no-go: pay on reset (Landauer)

Equipartition: each thermal degree of freedom carries \(\sim k_B T/2\) of energy — information per bit is tiny beside macroscopic power flow.

  • Example: \(70\,\mathrm{kW}\) engine at \(T \approx 370\,\mathrm{K}\)
  • Thermal throughput \(\sim 2P/(k_B T) \approx 3\times 10^{25}\) DOF per second
  • One bit per DOF \(\Rightarrow\) \(\sim 10^{25}\) bits/s — orders of magnitude above all human storage and communication
  • Carnot already caps efficiency; demon-style feedback cannot close the gap at this scale

Molecular machines operate where \(k_B T\) is the energy scale and thermal fluctuations are the environment — not a nuisance to fight.

  • ATP synthase: rotary motor driven by proton flow down the mitochondrial gradient
  • \(\sim 3\)–\(4\) protons per ATP; proton arrival is a discrete yes/no — a bit-scale measurement
  • Ratcheted Brownian motion: information about which side/proton phase drives rotation
  • Biology composes \(\sim 10^3\)–\(10^4\) such engines per cell; the brain’s ATP budget is built from them

Landauer’s Principle

Boltzmann Memory: Equal Double Well

Equal depth isolates information from energetics: \(E_L=E_R\), so \(p_L=p_R=\tfrac12\) at equilibrium.

  • Store: particle trapped in left or right basin (barrier \(\gg k_BT\))
  • Read: label the basins \(0\) and \(1\)
  • Erase: merge to one well — Landauer pays \(\ge k_BT\ln 2\) if the bit was unknown

Erasing an equiprobable bit compresses two basins into one: \(H\to 0\) costs at least \(k_BT\ln 2\).

  • Unknown bit: \(H(M)=\ln 2\) nats before reset
  • Known bit already in the target well: erasure can be free
  • Equal wells: no \(\Delta F\) between labels — pure information cost

A tape of equal-depth bits exchanges entropy, not energy, with the rest of the ledger.

  • One equal double well \(=\) one cell of an information reservoir
  • Many such cells \(=\) a tape whose Shannon entropy is a thermodynamic resource or debt
  • Boltzmann machine \(=\) a network of such bits with energy \(E(s)\) — week 5

Implications for Information Engines

  • Information engines must overcome thermal noise
  • Related threshold for:
    • Information erasure
    • Information transmission
  • Temperature sets fundamental noise floor

The demon measures; Szilard’s engine stores one bit. Neither violates the second law until the record is erased.

  • Landauer (1961): erasing one bit at \(T\) costs at least \(k_B T \ln 2\)
  • No-go: you cannot erase for free
  • Prescription: the demon’s policy — which molecules to let through

Information and Intelligence: First Cut

Further Reading

  • Szilard engine and demon resolution (logical irreversibility of erasure) of Bennett (1982)

  • introduction and Szilárd engine section of Parrondo et al. (2015)

  • physical memory; metastable states of Parrondo et al. (2015)

  • erasure cost of Landauer (1961)

  • information reservoirs (optional) of Barato and Seifert (2014)

  • the whole paper of Landauer (1961)

  • Section 3.3, pages 49–52 of Welling et al. (2026)

Thanks!

References

Barato, A.C., Seifert, U., 2014. Stochastic thermodynamics with information reservoirs. Physical Review E 90, 042150. https://doi.org/10.1103/PhysRevE.90.042150
Bennett, C.H., 1982. The thermodynamics of computation—–a review. International Journal of Theoretical Physics 21, 905–940.
Landauer, R., 1961. Irreversibility and heat generation in the computing process. IBM Journal of Research and Development 5, 183–191. https://doi.org/10.1147/rd.53.0183
Parrondo, J.M.R., Horowitz, J.M., Sagawa, T., 2015. Thermodynamics of information. Nature Physics 11, 131–139. https://doi.org/10.1038/nphys3230
Welling, M., Lu, S., Holdijk, L., 2026. Generative AI and stochastic thermodynamics: A tale of free energies. Cambridge University Press, Cambridge, U.K.