# Arbitrary Order Meta-Learning with Simple Population-Based Evolution

arXiv: https://arxiv.org/abs/2303.09478 · HTML: https://mlsanity.com/paper/2303.09478/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 23 references that are in the map.
- First posted on arXiv 16 Mar 2023.

## Builds on (most cited first, 11 of 11)

- [Discovered Policy Optimisation](https://mlsanity.com/paper/2210.05639/) (2022)
- [Gradients are Not All You Need](https://mlsanity.com/paper/2111.05803/) (2021)
- [Model-Free Opponent Shaping](https://mlsanity.com/paper/2205.01447/) (2022)
- [Discovering Evolution Strategies via Meta-Black-Box Optimization](https://mlsanity.com/paper/2211.11260/) (2022)
- [A Modern Self-Referential Weight Matrix That Learns to Modify Itself](https://mlsanity.com/paper/2202.05780/) (2022)
- [COLA: Consistent Learning with Opponent-Learning Awareness](https://mlsanity.com/paper/2203.04098/) (2022)
- [Towards a Theory of Evolution as Multilevel Learning](https://mlsanity.com/paper/2110.14602/) (2021)
- [Adversarial Cheap Talk](https://mlsanity.com/paper/2211.11030/) (2022)
- [Eliminating Meta Optimization Through Self-Referential Meta Learning](https://mlsanity.com/paper/2212.14392/) (2022)
- [Training Learned Optimizers with Randomly Initialized Learned Optimizers](https://mlsanity.com/paper/2101.07367/) (2021)
- [Population-Based Evolution Optimizes a Meta-Learning Objective](https://mlsanity.com/paper/2103.06435/) (2021)

## Built on by (most cited first, 4 of 4)

- [Automated Design of Agentic Systems](https://mlsanity.com/paper/2408.08435/) (2024)
- [Discovering Preference Optimization Algorithms with and for Large Language Models](https://mlsanity.com/paper/2406.08414/) (2024)
- [The Red Queen G"odel Machine: Co-Evolving Agents and Their Evaluators](https://mlsanity.com/paper/2606.26294/) (2026)
- [Artificial Generational Intelligence: Cultural Accumulation in Reinforcement Learning](https://mlsanity.com/paper/2406.00392/) (2024)

## Abstract (arXiv)

Meta-learning, the notion of learning to learn, enables learning systems to quickly and flexibly solve new tasks. This usually involves defining a set of outer-loop meta-parameters that are then used to update a set of inner-loop parameters. Most meta-learning approaches use complicated and computationally expensive bi-level optimisation schemes to update these meta-parameters. Ideally, systems should perform multiple orders of meta-learning, i.e. to learn to learn to learn and so on, to accelerate their own learning. Unfortunately, standard meta-learning techniques are often inappropriate for these higher-order meta-parameters because the meta-optimisation procedure becomes too complicated or unstable. Inspired by the higher-order meta-learning we observe in real-world evolution, we show that using simple population-based evolution implicitly optimises for arbitrarily-high order meta-parameters. First, we theoretically prove and empirically show that population-based evolution implicitly optimises meta-parameters of arbitrarily-high order in a simple setting. We then introduce a minimal self-referential parameterisation, which in principle enables arbitrary-order meta-learning. Finally, we show that higher-order meta-learning improves performance on time series forecasting tasks.