# Remember What You Did: Action-History Memory with Dual-Expert Denoising for Long-Horizon Vision-Language-Action Policies

arXiv: https://arxiv.org/abs/2609.37307 · HTML: https://mlsanity.com/paper/2609.37307/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Reasoning and chain-of-thought in VLA, one of 13 directions in Vision-language-action models.
- Builds on 29 of its 37 references that are in the map.
- First posted on arXiv 29 Sep 2026.

## Placed in

- Vision-language-action models / [Reasoning and chain-of-thought in VLA](https://mlsanity.com/vla/reasoning-and-chain-of-thought-in-vla/)
- State-space sequence models (S4, Mamba) / [Video, motion and generative Mamba models](https://mlsanity.com/ssm/video-motion-and-generative-mamba-models/)

## Builds on (most cited first, 29 of 29)

- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [π0: A Vision-Language-Action Flow Model for General Robot Control](https://mlsanity.com/paper/2410.24164/) (2024)
- [π0.5: a Vision-Language-Action Model with Open-World Generalization](https://mlsanity.com/paper/2504.16054/) (2025)
- [GR00T N1: An Open Foundation Model for Generalist Humanoid Robots](https://mlsanity.com/paper/2503.14734/) (2025)
- [Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success](https://mlsanity.com/paper/2502.19645/) (2025)
- [π0.7: a Steerable Generalist Robotic Foundation Model with Emergent Capabilities](https://mlsanity.com/paper/2604.15483/) (2026)
- [MEM: Multi-Scale Embodied Memory for Vision Language Action Models](https://mlsanity.com/paper/2603.03596/) (2026)
- [RoboMME: Benchmarking and Understanding Memory for Robotic Generalist Policies](https://mlsanity.com/paper/2603.04639/) (2026)
- [Xiaomi-Robotics-0: An Open-Sourced Vision-Language-Action Model with Real-Time Execution](https://mlsanity.com/paper/2602.12684/) (2026)
- [RMBench: Memory-Dependent Robotic Manipulation Benchmark with Insights into Policy Design](https://mlsanity.com/paper/2603.01229/) (2026)
- [ContextVLA: Vision-Language-Action Model with Amortized Multi-Frame Context](https://mlsanity.com/paper/2510.04246/) (2025)
- [ReMem-VLA: Empowering Vision-Language-Action Model with Memory via Dual-Level Recurrent Queries](https://mlsanity.com/paper/2603.12942/) (2026)
- [RoboMemArena: A Comprehensive and Challenging Robotic Memory Benchmark](https://mlsanity.com/paper/2605.10921/) (2026)
- [EventVLA: Event-Driven Visual Evidence Memory for Long-Horizon Vision-Language-Action Policies](https://mlsanity.com/paper/2606.20092/) (2026)
- [HoloBrain-0 Technical Report](https://mlsanity.com/paper/2602.12062/) (2026)
- [Global Prior Meets Local Consistency: Dual-Memory Augmented Vision-Language-Action Model for Efficient Robotic Manipulation](https://mlsanity.com/paper/2602.20200/) (2026)
- [History-Aware Visuomotor Policy Learning via Point Tracking](https://mlsanity.com/paper/2509.17141/) (2025)
- [Shallow-π: Knowledge Distillation for Flow-based VLAs](https://mlsanity.com/paper/2601.20262/) (2026)
- [Memory Retrieval in Visuomotor Policies for Long-Horizon Robot Control](https://mlsanity.com/paper/2606.25136/) (2026)
- [CycleManip: Enabling Cyclic Task Manipulation via Effective Historical Perception and Understanding](https://mlsanity.com/paper/2512.01022/) (2025)
- [StreamVLA: Breaking the Reason-Act Cycle via Completion-State Gating](https://mlsanity.com/paper/2602.01100/) (2026)
- [NativeMEM: Native Memory Compression for Long-Horizon Robotic Manipulation](https://mlsanity.com/paper/2607.06678/) (2026)
- [Beyond Short-Horizon: VQ-Memory for Robust Long-Horizon Manipulation in Non-Markovian Simulation Benchmarks](https://mlsanity.com/paper/2603.09513/) (2026)
- [KEMO: Event-Driven Keyframe Memory for Long-Horizon Robot Manipulation with VLA Policies](https://mlsanity.com/paper/2606.23589/) (2026)
- [Ambient Diffusion Policy: Imitation Learning from Suboptimal Data in Robotics](https://mlsanity.com/paper/2606.12365/) (2026)
- [Dual Latent Memory in Vision-Language-Action Models for Robotic Manipulation](https://mlsanity.com/paper/2607.07608/) (2026)
- [TacMamba: A Tactile History Compression Adapter Bridging Fast Reflexes and Slow VLA Reasoning](https://mlsanity.com/paper/2603.01700/) (2026)
- [Action-Effect Memory Pretraining for Robot Manipulation](https://mlsanity.com/paper/2606.12499/) (2026)
- [ChainVLA: Chaining Vision-Language-Action Queries through a Unified Execution State for Long-Horizon Manipulation](https://mlsanity.com/paper/2608.02326/) (2026)

## Abstract (arXiv)

Vision-language-action (VLA) models have driven rapid progress in robotic manipulation, demonstrating strong fine-grained control and promising performance on long-horizon tasks. However, many existing VLAs lack explicit access to interaction history, making them vulnerable to perceptual aliasing: similar current observations and robot states at different task stages may induce action ambiguity and lower success rate. Existing methods incorporate temporal or progress cues through feature conditioning, action-prior modification, or sampling guidance. However, methods that jointly fine-tune memory modules and the base VLA incur additional policy-training costs, motivating the separation of trainable history-conditioned steering from frozen base-policy refinement. We propose ActMem-VLA, a dual-expert handover architecture that augments a frozen, fine-tuned VLA with a memory plugin comprising a Mamba-based memory module and a lightweight PreAction Expert (PAE). Specifically, Mamba encodes executed-action history into memory that conditions PAE alongside current context. With these inputs, PAE steers task progression during early, high-noise denoising, then passes the partially denoised action to the frozen Action Expert (AE) to refine action details during the remaining low-noise steps. The fine-tuned base VLA remains frozen throughout training, while only the Mamba module and PAE are jointly optimized. On LIBERO-Mem, ActMem-VLA achieves 80.8% average success across all ten tasks, compared with 65.2% for π0.5 and 49.5% for MemoryVLA, while introducing only 3.45% additional parameters. Across four real-world tasks, it improves the average success rate over π0.5 by 28.8%.