# Minimal Recurrent Behavioral Memory for Imitation under Partial Observability

arXiv: https://arxiv.org/abs/2609.25757 · HTML: https://mlsanity.com/paper/2609.25757/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 37 of its 52 references that are in the map.
- First posted on arXiv 22 Sep 2026.

## Builds on (most cited first, 30 of 37)

- [RMA: Rapid Motor Adaptation for Legged Robots](https://mlsanity.com/paper/2107.04034/) (2021)
- [Learning Invariant Representations for Reinforcement Learning without Reconstruction](https://mlsanity.com/paper/2006.10742/) (2020)
- [Causal Confusion in Imitation Learning](https://mlsanity.com/paper/1905.11979/) (2019)
- [Behavior Generation with Latent Actions](https://mlsanity.com/paper/2403.03181/) (2024)
- [Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations](https://mlsanity.com/paper/1704.00648/) (2017)
- [Scalable methods for computing state similarity in deterministic Markov Decision Processes](https://mlsanity.com/paper/1911.09291/) (2019)
- [Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow](https://mlsanity.com/paper/1810.00821/) (2018)
- [Fighting Copycat Agents in Behavioral Cloning from Observation Histories](https://mlsanity.com/paper/2010.14876/) (2020)
- [Approximate information state for approximate planning and reinforcement learning in partially observed systems](https://mlsanity.com/paper/2010.08843/) (2020)
- [When Do Transformers Shine in RL? Decoupling Memory from Credit Assignment](https://mlsanity.com/paper/2307.03864/) (2023)
- [POPGym: Benchmarking Partially Observable Reinforcement Learning](https://mlsanity.com/paper/2303.01859/) (2023)
- [Bridging State and History Representations: Understanding Self-Predictive RL](https://mlsanity.com/paper/2401.08898/) (2024)
- [Guaranteed Discovery of Control-Endogenous Latent States with Multi-Step Inverse Models](https://mlsanity.com/paper/2207.08229/) (2022)
- [Memory, Benchmark & Robots: A Benchmark for Solving Complex Tasks with Reinforcement Learning](https://mlsanity.com/paper/2502.10550/) (2025)
- [Efficient Transformers in Reinforcement Learning using Actor-Learner Distillation](https://mlsanity.com/paper/2104.01655/) (2021)
- [Sequence Model Imitation Learning with Unobserved Contexts](https://mlsanity.com/paper/2208.02225/) (2022)
- [Learning Long-Context Diffusion Policies via Past-Token Prediction](https://mlsanity.com/paper/2505.09561/) (2025)
- [Learning Belief Representations for Imitation Learning in POMDPs](https://mlsanity.com/paper/1906.09510/) (2019)
- [Representation Learning with Multi-Step Inverse Kinematics: An Efficient and Optimal Approach to Rich-Observation RL](https://mlsanity.com/paper/2304.05889/) (2023)
- [Memory Gym: Towards Endless Tasks to Benchmark Memory Capabilities of Agents](https://mlsanity.com/paper/2309.17207/) (2023)
- [Rapid Motor Adaptation for Robotic Manipulator Arms](https://mlsanity.com/paper/2312.04670/) (2023)
- [Gated Memory Policy: In-Context Memorization and Adaptation](https://mlsanity.com/paper/2604.18933/) (2026)
- [Memory Retrieval in Visuomotor Policies for Long-Horizon Robot Control](https://mlsanity.com/paper/2606.25136/) (2026)
- [Benchmarking Partial Observability in Reinforcement Learning with a Suite of Memory-Improvable Domains](https://mlsanity.com/paper/2508.00046/) (2025)
- [Remember the Decision, Not the Description: A Rate-Distortion Framework for Agent Memory](https://mlsanity.com/paper/2605.10870/) (2026)
- [Learning Memory Mechanisms for Decision Making through Demonstrations](https://mlsanity.com/paper/2411.07954/) (2024)
- [The Myhill-Nerode Theorem for Bounded Interaction: Canonical Abstractions via Agent-Bounded Indistinguishability](https://mlsanity.com/paper/2603.21399/) (2026)
- [Minimum-Information LQG Control - Part II: Retentive Controllers](https://mlsanity.com/paper/1606.01947/) (2016)
- [Generalizing Multi-Step Inverse Models for Representation Learning to Finite-Memory POMDPs](https://mlsanity.com/paper/2404.14552/) (2024)
- [Training and Evaluating Diffusion Policies with Long Context Lengths](https://mlsanity.com/paper/2606.16447/) (2026)

## Abstract (arXiv)

What is the least recurrent memory needed to reproduce a specified expert under partial observability? The instantaneous requirement is the conditional entropy of the expert's behavioral quotient, but recurrence must also preserve distinctions that future observations will not restore before use. We characterize this minimal recurrent behavioral memory by a compatibility relation: under transitivity its classes attain the exact minimum, while the general case is an entropy minimization over closed compatible state assignments, with exact certificates on finite instances. A sole-carrier measurement protocol separates behavioral sufficiency, excess code rate, and information carried by observations or other memory paths; experimental bit requirements refer to the induced symbolic behavioral model under the stated occupancy. Across manipulation tasks, learned code rates remain near zero- and two-bit requirements as hidden modes grow to 512, and anticipatory memory follows a 2→1→0 requirement despite zero instantaneous demand during waiting. Learning this representation remains difficult: event-agnostic future-behavior supervision yields 36/40 sufficient seeds with one frozen configuration and improves the longest-horizon pixel setting from 0/8 to 6/8 sufficient held-out seeds (closed-loop success from 0.08 to 0.57). On unmodified community benchmarks, the protocol certifies delay-independent requirements, which sufficient codes match at mid-delay. The supervision aids commitment but can induce predictive surplus; annealing it lets imitation and rate training reduce that surplus, separating the information-theoretic target from the ability to learn it.