# Resolving State Ambiguity in Robot Manipulation via Adaptive Working Memory Recoding

arXiv: https://arxiv.org/abs/2512.24638 · HTML: https://mlsanity.com/paper/2512.24638/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Dec 2025.

## Built on by (most cited first, 6 of 6)

- [ReMem-VLA: Empowering Vision-Language-Action Model with Memory via Dual-Level Recurrent Queries](https://mlsanity.com/paper/2603.12942/) (2026)
- [Gated Memory Policy: In-Context Memorization and Adaptation](https://mlsanity.com/paper/2604.18933/) (2026)
- [Dual Latent Memory in Vision-Language-Action Models for Robotic Manipulation](https://mlsanity.com/paper/2607.07608/) (2026)
- [ChainVLA: Chaining Vision-Language-Action Queries through a Unified Execution State for Long-Horizon Manipulation](https://mlsanity.com/paper/2608.02326/) (2026)
- [Skills in Weights, Memory in Code: Hybrid Learning for Memory-Dependent Robot Manipulation](https://mlsanity.com/paper/2608.09410/) (2026)
- [PMTRM: Pseudo-Memory Temporal Re-encoding Module for Embodied Policy Learning](https://mlsanity.com/paper/2610.11168/) (2026)

## Abstract (arXiv)

State ambiguity is common in robotic manipulation. Identical observations may correspond to multiple valid behavior trajectories. The visuomotor policy must correctly extract the appropriate types and levels of information from the history to identify the current task phase. However, naively extending the history window is computationally expensive and may cause severe overfitting. Inspired by the continuous nature of human reasoning and the recoding of working memory, we introduce PAM, a novel visuomotor Policy equipped with Adaptive working Memory. With minimal additional training cost in a two-stage manner, PAM supports a 300-frame history window while maintaining high inference speed. Specifically, a hierarchical frame feature extractor yields two distinct representations for motion primitives and temporal disambiguation. For compact representation, a context router with range-specific queries is employed to produce compact context features across multiple history lengths. And an auxiliary objective of reconstructing historical information is introduced to ensure that the context router acts as an effective bottleneck. We meticulously design 7 tasks and verify that PAM can handle multiple scenarios of state ambiguity simultaneously. With a history window of approximately 10 seconds, PAM still supports stable training and maintains inference speeds above 20Hz. Project website: https://tinda24.github.io/pam/