# Memory Retrieval in Visuomotor Policies for Long-Horizon Robot Control

arXiv: https://arxiv.org/abs/2606.25136 · HTML: https://mlsanity.com/paper/2606.25136/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jun 2026.

## Built on by (most cited first, 8 of 8)

- [TraceFlow: Guiding Frozen Flow-Matching Robot Policies with Success and Failure Traces](https://mlsanity.com/paper/2609.20646/) (2026)
- [In-Context Learning for Robots: Methods and Applications](https://mlsanity.com/paper/2609.36012/) (2026)
- [WorldToken: Time-First Sequence Modeling for Robotic Imitation Learning](https://mlsanity.com/paper/2608.22591/) (2026)
- [StenoVLA-3D: 3D-Aware Reasoning VLA for Navigation Through Gastrointestinal Stenoses](https://mlsanity.com/paper/2609.24187/) (2026)
- [Minimal Recurrent Behavioral Memory for Imitation under Partial Observability](https://mlsanity.com/paper/2609.25757/) (2026)
- [ECHO: Event-Augmented Context with Hindsight and Outlook for Wrist-Only Manipulation](https://mlsanity.com/paper/2609.34893/) (2026)
- [ReCAT: Remember, Count, and Time: Structured Recurrent Memory for Robot Manipulation](https://mlsanity.com/paper/2609.35200/) (2026)
- [Remember What You Did: Action-History Memory with Dual-Expert Denoising for Long-Horizon Vision-Language-Action Policies](https://mlsanity.com/paper/2609.37307/) (2026)

## Abstract (arXiv)

General-purpose robots operating in partially observable environments, such as homes, require memory to support autonomy. They must recall diverse information from the past, such as where objects were placed, which tasks a human partner has completed, and when an appliance was turned on. Achieving this versatility requires a general memory retrieval mechanism. Transformer architectures that use attention over long contexts for memory retrieval provide a promising approach, as they learn retrieval from data rather than relying on task-specific or hand-designed rules. However, directly incorporating them into imitation learning from offline data introduces two key challenges: (1) the policy may learn spurious correlations between past information and predicted actions, and (2) errors accumulate in memory due to prediction inaccuracies and their compounding interactions with the environment, causing model drift and cascading failures. To address both challenges, we introduce HALO, a visuomotor policy with an attention-based memory retrieval mechanism for long-horizon control. First, to suppress spurious correlations, HALO distills vision-language model (VLM) priors into the policy. It generates memory-dependent question--answer pairs from demonstration trajectories and trains jointly with a video question--answering objective, steering retrieval toward task-relevant information. Second, to reduce the impact of accumulated errors in memory during closed-loop control, HALO uses sparse attention that restricts retrieval to only the most relevant parts of the history. Together, these components enable more reliable long-horizon control by guiding the policy to retrieve task-relevant information from up to eight minutes of past experience. Project website: https://robin-lab.cs.utexas.edu/HALO