mlsanityData to 14 Oct 2026

The Need for a Big World Simulator: A Scientific Challenge for Continual Learning

arXiv 2408.02930 ↗First posted 6 Aug 2024cs.LG cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

The "small agent, big world" frame offers a conceptual view that motivates the need for continual learning. The idea is that a small agent operating in a much bigger world cannot store all information that the world has to offer. To perform well, the agent must be carefully designed to ingest, retain, and eject the right information. To enable the development of performant continual learning agents, a number of synthetic environments have been proposed. However, these benchmarks suffer from limitations, including unnatural distribution shifts and a lack of fidelity to the "small agent, big world" framing. This paper aims to formalize two desiderata for the design of future simulated environments. These two criteria aim to reflect the objectives and complexity of continual learning in practical settings while enabling rapid prototyping of algorithms on a smaller scale.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 19 of its 28 references that are in the map. First posted on arXiv 6 Aug 2024.

Builds on

19 of 28 references.

References · 19

  1. 2020Scaling Laws for Neural Language Modelscited by 6,586
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  3. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  4. 2017Gradient Episodic Memory for Continual Learningcited by 1,898
  5. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,375
  6. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 722
  7. 2017Variational Continual Learningcited by 339
  8. 2017CORe50: a New Dataset and Benchmark for Continuous Object Recognitioncited by 289
  9. 2023Understanding plasticity in neural networkscited by 174
  10. 2019Behaviour Suite for Reinforcement Learningcited by 132
  11. 2023Loss of Plasticity in Continual Deep Reinforcement Learningcited by 121
  12. 2022TemporalWiki: A Lifelong Benchmark for Training and Evaluating Ever-Evolving Language Modelscited by 119
7 more
  1. 2023A Definition of Continual Reinforcement Learningcited by 101
  2. 2022Wild-Time: A Benchmark of in-the-Wild Distribution Shift over Timecited by 90
  3. 2021Continual Backprop: Stochastic Gradient Descent with Persistent Randomnesscited by 80
  4. 2021Simple Agent, Complex Environment: Efficient Reinforcement Learning with Agent Statescited by 43
  5. 2023Maintaining Plasticity in Deep Continual Learningcited by 43
  6. 2023Continual Learning as Computationally Constrained Reinforcement Learningcited by 37
  7. 2021The Neural Testbed: Evaluating Joint Predictionscited by 22

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 2.

5 papers.

  1. 2025RLVE: Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable Environmentscited by 50
  2. 2026Golden Goose: A Simple Trick to Synthesize Unlimited RLVR Tasks from Unverifiable Internet Textcited by 10
  3. 2025Memory Allocation in Resource-Constrained Reinforcement Learningcited by 2
  4. 2025The World Is Bigger! A Computationally-Embedded Perspective on the Big World Hypothesiscited by 1
  5. 2026Position: Deployed Reinforcement Learning should be Continualcited by 1