Overview
The seminal paper Learning to Learn by Gradient Descent By Gradient Descent [1] explores learning optimisers (rather than using SGD or Adam) through an RNN architecture which models time steps as training steps and an RL based loss.
In this project we aim to achieve a similar product but rather than RNNs and RL we would like to explore the closely linked diffusion models [5,6] and stochastic control. In particular there is already theoretical work that motivates the use of these methodologies for global optimisation [2], what now remains is to explore it in practice.
We expect the student to lightly adapt the methods in [3,4] to the optimisation setting in [2] (This simply amounts to exploring low temperatures in the artificial target distribution induced by the loss function). Furthermore, exploring tasks in engineering as in [1] might require being creative about the inductive biases baked into the NN parametrisation that we are working with.
The nature of this project will be mostly ML-engineering and playing around / being creative about NN inductive biases. That said if the student is more mathematically/theory oriented we could explore extending the results in [2] (which apply to the method in [4] only) to the method in in [3] using the mixing rates in [7] and maybe comparing to things like [8] (last bit would be very bonus/extra type of thing).
As always the student should explore simple 1d and 2d toy optimisation examples to assess the validity of the method before moving onto real world examples.
[1] https://arxiv.org/pdf/1606.04474.pdf
[2] https://arxiv.org/abs/2111.00402
[3] https://openreview.net/forum?id=8pvnfTAbu1f
[4] https://arxiv.org/abs/2111.15141
[5] https://arxiv.org/abs/2006.11239
[6] https://arxiv.org/abs/2006.11239
