Week 7: Good Regulator, Goals, and Channels

[jupyter][google colab][reveal][edit]

Neil D. Lawrence

Abstract:

Open with an unmarked review of Worksheet 3 submissions, then develop the Good Regulator overview Week 6 did not reach: D, S, A, Z; minimise H(Z); existence of a deterministic H(A|S)=0 policy; two caveats. Map that criterion onto a goal measure via Radon–Nikodym (reward as −ln(dν/dλ)), bridging Week 5’s relative-entropy language. Recover the unread Week 6 overflow: Shannon capacity R≤C and the data-processing inequality as statement and moral. Multi-information, the DPI proof, and the information bottleneck stay notes overflow for Week 8.

No class test today. Quiz 3 was last week. Worksheet 3 was due last week; Worksheet 4 (Skepticus dialogue on GRT and general intelligence) is already out (due 1 December, before Quiz 4). As taught in Week 6: exponential family / Fisher / natural gradient; Shannon capacity, DPI, and \(D,S,A,Z\) diagrams were not walked (GRT was a two-minute W4 pointer). Today: unmarked W3 review; GRT overview; goal measure via Radon–Nikodym; Shannon \(R\le C\) and DPI statement.

This Session

Time plan. The slot is 10:05–11:55 (110 minutes, not 120). Do not rebuild Fisher geometry or natural gradient. Do not board the full Conant–Ashby concavity transfer \(g(\alpha)\). Multi-information / DPI proof / IB are notes overflow — not today’s clock.

ClockBlock
10:05–10:25Unmarked Worksheet 3 review; flag W4 due Week 8 start
10:25–10:50Good Regulator overview: \(D,S,A,Z\); \(\min H(Z)\); existence slogan; caveats
10:50–11:00Break
11:00–11:35Goal measure / Radon–Nikodym; reward as \(-\ln(\mathrm{d}\nu/\mathrm{d}\lambda)\)
11:35–11:50Shannon \(R\le C\); DPI statement and moral
11:50–11:55Define-this-week; W4 reminder; Week 8 preview

Information, energy and intelligence course notebook setup

[edit]

We install some bespoke code for creating and saving plots as well as loading data sets.

import importlib.util
cmd = install_command('pods')
%system {cmd}
cmd = install_command('notutils')
%system {cmd}
cmd = install_command('mlai')
%system {cmd}

Submission Review

[edit]

Class test timer (Worksheet 3 review): 10:05

Worksheet 3 asked for a Skepticus dialogue on coarse-graining Maxwell. Review patterns only: where dialogues kept curiosity then skepticism apart; where models collapsed the demon into a single slogan; where students pressed compartments and two ledgers usefully. Do not announce marks. Flag Worksheet 4 due at the start of lecture 8 (1 December), before Quiz 4.

Good Regulator Overview

Class test timer (Good Regulator): 10:25

Week 6 reached only a two-minute W4 pointer. Today is the first live \(D,S,A,Z\) overview: diagrams, \(\min H(Z)\), existence of a deterministic \(H(A\mid S)=0\) policy named (not proved via \(g(\alpha)\)), and both caveats. Week 8 will not re-prove Conant–Ashby; it moves to limits synthesis and quantum motivation.

Good Regulator Overview

[edit]

Overview only. Full Conant–Ashby development — concavity argument, existence of a deterministic optimum, and the two caveats in depth — is the Week 8 review. Today’s job is the setup, the notation, and the slogan and its two caveats. Diagrams follow Joery’s oneshot figure in the good-regulator notes: disturbance \(D\), system state \(S\), action \(A\), reguland \(Z\).

Conant and Ashby (Conant and Ashby, 1970) often label the regulator \(R\). We write \(A\) for the action so the policy \(\pi(a\mid s)\) is first-class. The reguland — the variable whose entropy successful regulation minimises — is \(Z\). The system state is \(S\); the disturbance is \(D\). The cumulant generating function earlier today was \(\psi(\boldsymbol{\theta})=\log Z(-\boldsymbol{\theta})\); that \(Z\) was the partition function, not the reguland — different objects, same letter in the physics tradition.

Figure: Cause-controlled regulation (left) acts on the disturbance \(D\); error-controlled regulation (right) acts on the state \(S\) that \(D\) caused.

A disturbance \(D\) causes events in a system \(S\); the regulator counters with an action \(A\); the pair produces a reguland \(Z=\phi(S,A)\) (Conant and Ashby, 1970). Cause-controlled regulation acts on \(D\) (or on an inferred model of \(D\)). Error-controlled regulation acts on the state \(S\) caused by \(D\), so by the data-processing inequality \(I(A;D)\le I(S;D)\). Both diagrams are shown above; the existence proof that an \(H(Z)\)-optimal policy may be taken deterministic waits for Week 8.

Successful regulation, for Conant and Ashby, minimises the entropy of the reguland: \(\min_\pi H(Z)\). Among optima there exists a deterministic policy \(A=h(S)\), equivalently \(H(A\mid S)=0\). That is the sense in which the regulator is a model'' --- deliberately weak. Do not collapse the slogan into\(A\) contains a rich model of \(S\)’’: a constant action also has \(H(A\mid S)=0\). Do not confuse low \(H(Z)\) with a desirable outcome. Requisite variety in Shannon form will use the same letters: residual uncertainty satisfies \(H(Z)\ge H(D)-I(D;A)\) when \(A\) is the regulatory response. Week 8 reviews the theorem properly and applies it to capability claims.

Ashby’s law of requisite variety in Shannon form (Ashby, 1956), written in today’s letters: residual uncertainty in the reguland is at least the disturbance entropy minus the information the action carries about the disturbance. Requisite variety asks how much relevant \(I\) is required; the Good Regulator Theorem asks how that \(I\) may be organised. Full development of both, with the concavity argument, is Week 8.

The Good Regulator Theorem, in entropy form, is an existence result: among policies that minimise \(H(Z)\) there is a deterministic one, \(A=h(S)\), equivalently \(H(A\mid S)=0\) (Conant and Ashby, 1970). The full concavity / extreme-point argument is in the notes include; today the diagrams and the slogan-with-fences are enough for assessment. Requisite variety remains the quantity bound \(H(Z)\ge H(D)-I(D;A)\), not the structure theorem.

Class test timer (Break): 10:50

Goals as Radon–Nikodym

Class test timer (Goal measure): 11:00

Conant and Ashby minimise the entropy of the reguland. In the reinforcement-learning reading developed with Joery, the long-run state–action occupancy is pushed through a fixed channel \(K(z\mid s,a)\) to an outcome law \(p_\pi\) on \(Z\). Minimising \(H(Z)\) is concave over the occupancy polytope, so a deterministic policy is optimal — the same structural reason as an ordinary MDP return, with a different objective.

When the ``good’’ outcomes are not the whole alphabet, introduce a goal measure \(\nu\) absolutely continuous with respect to a base \(\lambda\). Entropy relative to \(\nu\) differs from entropy relative to \(\lambda\) by the goal term \[ \mathbb{E}\bigl[\ln\tfrac{\mathrm{d}\nu}{\mathrm{d}\lambda}(Z)\bigr]=-\mathbb{E}[r(Z)],\qquad r=-\ln\tfrac{\mathrm{d}\nu}{\mathrm{d}\lambda}. \] So ordinary reward maximisation is entropy minimisation under a tilted reference — the same change-of-measure grammar as Week 5’s KL / Radon–Nikodym block. Conant and Ashby’s good set \(G\) is the special case that puts smaller \(\nu\)-volume on \(G\) than on \(G^c\), so the goal term pays for landing in \(G\).

This is a course-level vocabulary map, not the Joery paper. Assessed: letters \(D,S,A,Z\); \(\min H(Z)\); \(H(A\mid S)=0\) with both caveats; \(r=-\ln(\mathrm{d}\nu/\mathrm{d}\lambda)\) as the goal/reward bridge to Week 5. Not assessed: majorisation–minimisation, empowerment as a summand, DeepSea thresholds.

Shannon Capacity and DPI

Class test timer (Channels): 11:35

Week 6 planned Shannon capacity and DPI after Fisher; as taught they were not reached. State both today. Proof of DPI and multi-information / \(I+H=C\) stay notes overflow for Week 8 enrichment.

Scaffolding, Not Outcomes

[edit]

The chain rule \(H(X,Y)=H(X)+H(Y|X)\) is the algebraic source of multi-information. Mutual information \(I(X;Y)=H(X)-H(X|Y)\) is the pairwise case; we do not yet treat \(n>2\). Channel capacity \(C\) is a no-go on rate; the capacity-achieving input distribution is the prescription. The data-processing inequality is the no-go on \(I\): processing cannot create information. Cover and Thomas Theorem 2.8.1 is the reading; the proof waits for week 8, when \(I\) is first-class.

Figure: Mutual information for a binary symmetric channel; capacity is achieved at uniform input.

The data-processing inequality is the no-go on \(I\): further processing of \(Y\) cannot increase information about \(X\). Week 5’s soft-partition leftover is the same grammar. Cover–Thomas 2.8.1 is proved in the notes overflow below; Quiz 4 examines the statement and moral, not the two expansions.

Notes Overflow: Multi-Information, DPI Proof, IB

The following blocks were the previous Week 7 live spine. They remain available as notes and as Week 8 enrichment if the room wants depth after Quiz 4. They are not on today’s clock and not on Quiz 4 except insofar as DPI’s statement was taught above.

Optional overflow: Multi-Information

Figure: Samples from independent Gaussian variables that represent horizontal and vertical velocities when our system is at equilibrium.

Sampling Two Dimensional Variables

[edit]

Figure: Samples from correlated Gaussian variables that represent vertical and horizontal velocity.

Figure: Two independent Gaussians for the \(x\) and \(y\) velocity of a ball.

Figure: A correlated Gaussian for the \(x\) and \(y\) velocity of a ball. If all balls were correlated in this way, this would imply that the whole box is moving towards the upper right or bottom left.

Figure: An anti-correlated Gaussian for the \(x\) and \(y\) velocity of a ball. If all balls were anti-correlated in this way, this would imply that the whole box is moving towards the upper left or bottom right.

The Classical Observer

[edit]

Figure: Here the observer is monitoring the movements of the particles. We’ve plotted the velocities alongside the 1 standard deviation contour of their theoretical distribution.

The Classical Observer - Correlated

Figure: Again the observer is monitoring the movements of the particles, but here their motion is correlated (\(\rho=0.95\)).

The Classical Observer - Anti-correlated

Figure: Here the observer is monitoring the movements of the particles, but here their motion is anti-correlated (\(\rho=-0.95\)).

Independent, correlated, and anti-correlated velocities can share the same one-dimensional Gaussians. Multi-information is the leftover \(I=\sum h_i-H\). With fixed marginal entropies \(C=\sum_i h_i\), conservation \(I+H=C\).

Figure: Multi-information and joint entropy for uniform-marginal binary pairs; \(I+H=C\) with \(C=2\).

Optional overflow: DPI Proof and Information Bottleneck

Data-Processing Inequality

[edit]

Mutual information is the pairwise case of multi-information: \(I(X;Y)=H(X)-H(X|Y)=H(Y)-H(Y|X)=H(X)+H(Y)-H(X,Y)\). Week 2 defined it from the chain rule and stated the data-processing inequality. We prove it now, because this week’s object is \(I\).

The data-processing inequality (Cover and Thomas, 1991, Thm 2.8.1) is the no-go on \(I\). If \(X\to Y\to Z\) is Markov, then \(I(X;Z)\le I(X;Y)\). The first expansion of \(I(X;Y,Z)\) uses the chain rule and drops \(I(X;Z\mid Y)\) by the Markov property. The second expansion is at least \(I(X;Z)\) because conditional mutual information is non-negative. Equality holds when \(I(X;Y\mid Z)=0\) — \(Y\) is a sufficient statistic of \(X\) for \(Z\), or \(Z\) already captures everything \(Y\) knew about \(X\).

This is the same grammar as \(I+H=C\). Conservation says you cannot have both high stored correlation and high free uncertainty without bound. Data processing says you cannot increase stored correlation by further processing. A claim that an intelligent system will “just process more” and thereby recover information that a Markov stage has already destroyed is a DPI violation.

KL Decay and Grouping

[edit]

\[ \text{KL}\left( p\,\|\,\pi \right)=\text{KL}\left( w\,\|\,w_\pi \right)+\sum_i w_i\, \text{KL}\left( p_i\,\|\,\pi_i \right) \]

For a Markov process with stationary distribution \(\pi\), \(\text{KL}\left( p_t\,\|\,\pi \right)\) is nonincreasing — the data-processing inequality applied to the evolving law. (Physics literature often calls the same functional relative entropy.) Decompose \(p\) into group weights \(w\) and conditionals \(p_i\), and likewise \(\pi\) into \(w_\pi\) and \(\pi_i\). Then \[\begin{align} \text{KL}\left( p\,\|\,\pi \right)=\text{KL}\left( w\,\|\,w_\pi \right)+\sum_i w_i\, \text{KL}\left( p_i\,\|\,\pi_i \right). \end{align}\] The first term is how far the macro weights sit from equilibrium weights; the second is residual non-equilibrium inside groups. Local equilibrium is the regime \(\text{KL}\left( p_i\,\|\,\pi_i \right)\approx 0\) with \(\text{KL}\left( w\,\|\,w_\pi \right)\) still free to fall. Under deterministic Hamiltonian flow the fine-grained Shannon entropy of the phase-space density is conserved (Liouville); the Gibbs–Ehrenfest account of the second law is that only a entropy increases. Week 2’s die made the static split; this is the same algebra along a trajectory toward equilibrium.

Information engines and finite-time protocols act primarily on the between-group term — changing which macrostate or which bit is occupied — while assuming or enforcing fast decay of the within-group divergences. That is why week 2’s stationary examples and week 6’s paths on the equilibrium manifold belong to one story.

Information Bottleneck

[edit]

The data-processing inequality forbids creating information by processing. It does not say which information to keep. The information bottleneck (Tishby et al., 1999) is the prescription: given an observation \(X\) and a relevance variable \(Y\), choose a representation \(T\) of \(X\) that keeps \(I(T;Y)\) and discards the rest of \(I(X;T)\).

The information bottleneck of Tishby, Pereira and Bialek is a constrained optimisation on mutual information. The encoder \(p(t\mid x)\) is Markov given \(X\), so \(Y \leftrightarrow X \leftrightarrow T\). Data processing then supplies the bound \(I(T;Y)\le I(X;Y)\): no representation of \(X\) can be more informative about \(Y\) than \(X\) itself. The Lagrange multiplier \(\beta\) interpolates. Small \(\beta\) spends almost no \(I\) on the representation. Large \(\beta\) recovers \(T=X\). The self-consistent encoder is an exponential family, \[ p(t\mid x) \propto p(t)\,\exp\bigl(-\beta\,\mathrm{KL}\bigl(p(y\mid x)\,\|\,p(y\mid t)\bigr)\bigr), \] the same Boltzmann form as week 4, now with a KL sufficient statistic. We do not run Blahut–Arimoto. The point is the pair: DPI is the no-go; the bottleneck is how you spend \(I\).

Week 4 asked for the least-committal \(p\) given moments. Here the constraint is a mutual information. Same Lagrange move, different constraint. Later in this session the bottleneck evaluates intelligence claims: a system that keeps only what is relevant for action is making an IB statement. The Good Regulator existence claim \(H(A\mid S)=0\) comes from concavity of entropy; only then may one offer the control interpretation \(\min I(S;A)\) subject to \(H(Z)\le\epsilon\) as an IB-shaped reading — an interpretation after the theorem, not the theorem itself. Requisite variety and the Good Regulator are tools we use in Week 8; the information bottleneck itself is the relevant-compression theorem.

IB as a gloss after GRT: \(\min I(S;A)\) subject to \(H(Z)\le\epsilon\) is a course interpretation, not Conant and Ashby’s equation. Optional LO9 name if ahead: MaxEnt as \(m\)-projection (geometric reading of Week 5; stated, not computed).

The Inaccessible Game

[edit]

We call our framework the “inaccessible game” because the system is isolated from external observation: an outside observer cannot extract or inject information, making the game’s internal state inaccessible.

Like other zero-player games, such as Conway’s Game of Life (Gardner, 1970), the system evolves according to internal rules without external interference. But unlike cellular automata where rules are chosen by design, in our game the rules emerge from an information-theoretic principle.

Why “Inaccessible?”

The game is inaccessible because our fourth axiom—information isolation—ensures that an external observer learns nothing. Recall from Baez-Fritz-Leinster that information gained from observing a process \(f: p \rightarrow q\) is measured by entropy change: \[ F(f) = H(p) - H(q). \]

But if marginal entropies are conserved (\(\sum h_i = C\)), then an external observer sees:

  • Before observation: \(\sum h_i = C\)
  • After observation: \(\sum h_i = C\)
  • Information gained: \(\Delta(\sum h_i) = 0\)

The observer learns nothing! The system is informationally isolated from the outside world.

What Makes It a Game?

The “game” has the following characteristics:

Players: None (zero-player game)

State: A probability distribution over variables, parametrized by natural parameters \(\boldsymbol{\theta}\)

Rules: Evolve to maximize entropy production subject to the constraint \(\sum h_i = C\)

Dynamics: Emerge from information geometry (Fisher information) and the constraint structure

The game starts in a maximally correlated state (high multi-information \(I\), low joint entropy \(H\)) and evolves toward states of higher entropy. Since \(I + H = C\), this means the system breaks down correlations (\(I\) decreases) as entropy increases.

Connection to Physical Reality

Why should we care about this abstract game? Because its dynamics exhibit structure that connects to fundamental physics:

  1. GENERIC structure: The dynamics decompose into reversible (conservative) and irreversible (dissipative) components
  2. Energy-entropy equivalence: In the thermodynamic limit, our marginal entropy constraint becomes equivalent to energy conservation
  3. Landauer’s principle: Information erasure necessarily involves the dissipative part of the dynamics

The inaccessible game provides an information-theoretic foundation for thermodynamic principles. It suggests that physical laws might emerge from information-theoretic constraints, rather than information theory being derived from physics.

Define This Week

After This Lecture

No class test today. Quiz 4 is 1 December at the start of lecture 8: Weeks 6–7 as delivered (Fisher from Week 6; GRT overview / goal measure / Shannon / DPI from today). Week 8 after the quiz is unassessed: short limits synthesis, then no-barber motivation for von Neumann / quantum information, then course close.

Further Reading

  • the Good Regulator Theorem of Conant and Ashby (1970)

  • requisite variety; Shannon form of Ashby (1956)

  • the Good Regulator Theorem of Conant and Ashby (1970)

  • requisite variety of Ashby (1956)

  • noisy channel coding / capacity of Shannon (1948)

  • Chapter 7 of Cover and Thomas (1991)

  • Theorem 2.8.1 of Cover and Thomas (1991)

  • the method; optional of Tishby et al. (1999)

Thanks!

For more information on these subjects and more you might want to check the following resources.

References

Ashby, W.R., 1956. An introduction to cybernetics. Chapman & Hall, London.
Conant, R.C., Ashby, W.R., 1970. Every good regulator of a system must be a model of that system. International Journal of Systems Science 1, 89–97.
Cover, T.M., Thomas, J.A., 1991. Elements of information theory. Wiley, New York.
Gardner, M., 1970. Mathematical games: The fantastic combinations of John Conway’s new solitaire game “life.” Scientific American 223, 120–123.
Shannon, C.E., 1948. A mathematical theory of communication. The Bell System Technical Journal 27, 379–423. https://doi.org/10.1002/j.1538-7305.1948.tb01338.x
Tishby, N., Pereira, F.C., Bialek, W., 1999. The information bottleneck method, in: Proceedings of the 37th Annual Allerton Conference on Communication, Control and Computing. pp. 368–377.