What Must Generalist Agents Remember?
Not on the map. Why?
Abstract (arXiv)
This paper develops a formal account of what generalist agents must store in memory in order to act near-optimally across multiple environments and goals. It shows that when two domains share an observational bottleneck but require incompatible optimal actions, any uniformly near-optimal policy must induce distinct memory distributions at that bottleneck. The result yields a separation theorem: sufficiently successful agents cannot rely only on current state observations, but must preserve domain-relevant information in memory. The paper further shows that if an agent's memory contains enough information to estimate values for related goals, then that memory can be used to approximately reconstruct the agent's local transition dynamics. Together, these results characterize memory as the substrate that supports domain disambiguation, transition-model reconstruction, and planning for generalist agents.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 7 of its 18 references that are in the map. First posted on arXiv 17 Jun 2026.
Builds on
7 of 18 references.
References · 7
- 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 503
- 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 465
- 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 389
- 2019Provably efficient RL with Rich Observations via Latent State Decodingcited by 169
- 2020Invariant Causal Prediction for Block MDPscited by 105
- 2024Robust agents learn causal world modelscited by 81
- 2013Hidden Parameter Markov Decision Processes: A Semiparametric Regression Approach for Discovering Latent Task Parametrizationscited by 70
Built on by
Citing papers per year: 2026: 1.
1 paper.