Linear Recurrent Memory Suffices to Distil a World-Model Policy for Robot Air Hockey
On the map: World models → Latent dynamics models and model-based RL also fits: Latent predictive world models
Abstract (arXiv)
Does memory-dependent control need nonlinear recurrent dynamics? We study simulated air-hockey defence under temporary loss of puck tracking. A DreamerV3 teacher outperforms a memoryless policy under tracking loss, while resetting the teacher's recurrent state sharply reduces performance, which demonstrates that the task requires memory. We distil this teacher into compact recurrent policies with a 64 dimensional state, with a combination of a diagonal linear recurrence and an optional rank-k nonlinear innovation while retaining nonlinear observation encoders and action heads. Across five matched seeds, the purely linear recurrent model (k=0) matches both the GRU baseline and the teacher throughout the tested range of tracking loss. Increasing nonlinear innovation rank providing no measured benefits. This result is obtained on a fresh test split, which will be only opened after all models and analyses are frozen. The linear model requires fewer recurrent parameters and less computation than GRU, but performs comparably. These results suggest that, for this memory dependent control task, nonlinear representation learning around a simple linear memory mechanism can be sufficient, and that nonlinear recurrent dynamics are not necessarily required. These conclusions are limited to the simulated task, teacher, state dimension, and blackout horizon considered here, and to policies whose observation encoder and action head remain nonlinear.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 13 of its 14 references that are in the map. First posted on arXiv 30 Sep 2026.
Builds on
13 of 14 references.
Closest ancestors · 1
Same direction, newest first
Other references · 12
- 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
- 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 494
- 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 457
- 2015Policy Distillationcited by 414
- 2023Resurrecting Recurrent Neural Networks for Long Sequencescited by 397
- 2022Learning robust perceptive locomotion for quadrupedal robots in the wildcited by 299
- 2023Structured State Space Models for In-Context Reinforcement Learningcited by 109
- 2023POPGym: Benchmarking Partially Observable Reinforcement Learningcited by 59
- 2017Linking connectivity, dynamics and computations in low-rank recurrent neural networkscited by 51
- 2018Efficient Time-Series Approximation with Linear Recurrent Neural Networks: Architecture Learning and Predictive Powercited by 5
- 2024Learning to Play Air Hockey with Model-Based Deep Reinforcement Learningcited by 2
- 2024A Retrospective on the Robot Air Hockey Challenge: Benchmarking Robust, Reliable, and Safe Learning Techniques for Real-world Roboticscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026MemBodied: Recurrent Associative Memory for Vision-Language-Action Models
- 2026Can Vision-Language-Action Models Learn from Real-World Data Continually without Forgetting?
- 2026ReCAT: Remember, Count, and Time: Structured Recurrent Memory for Robot Manipulation
- 2026Scaling Short-Term Memory of Visuomotor Policies for Long-Horizon Tasks
- 2026T2Mem: Learning Test-Time Memory for Robotics ↗
- 2026ECoMEM: Explicit Concept Memory for Memory-Dependent Robot Control ↗
- 2026Chameleon: Control-Indexed Prospective Memory for Visuomotor Manipulation
- 2026MIKASA-Robo-VLA: Benchmarking Memory in VLA Models for Long-Horizon Manipulation