mlsanityData to 7 Oct 2026

MEM: Multi-Scale Embodied Memory for Vision Language Action Models

arXiv 2603.03596 ↗First posted 4 Mar 2026cs.RO cs.LGcited by 55

On the map: Vision-language-action models → Reasoning and chain-of-thought in VLA

Abstract (arXiv)

Conventionally, memory in end-to-end robotic learning involves inputting a sequence of past observations into the learned policy. However, in complex multi-stage real-world tasks, the robot's memory must represent past events at multiple levels of granularity: from long-term memory that captures abstracted semantic concepts (e.g., a robot cooking dinner should remember which stages of the recipe are already done) to short-term memory that captures recent events and compensates for occlusions (e.g., a robot remembering the object it wants to pick up once its arm occludes it). In this work, our main insight is that an effective memory architecture for long-horizon robotic control should combine multiple modalities to capture these different levels of abstraction. We introduce Multi-Scale Embodied Memory (MEM), an approach for mixed-modal long-horizon memory in robot policies. MEM combines video-based short-horizon memory, compressed via a video encoder, with text-based long-horizon memory. Together, they enable robot policies to perform tasks that span up to fifteen minutes, like cleaning up a kitchen, or preparing a grilled cheese sandwich. Additionally, we find that memory enables MEM policies to intelligently adapt manipulation strategies in-context.

Key facts. Cited by 55 papers in the map (data through 7 Oct 2026). Placed in Reasoning and chain-of-thought in VLA, one of 13 directions in Vision-language-action models. Builds on 45 of its 54 references that are in the map. First posted on arXiv 4 Mar 2026.

Builds on

45 of 54 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Rethinking Progression of Memory State in Robotic Manipulation: An Object-Centric Perspective14 Nov
  2. 2025MemoryVLA: Perceptual-Cognitive Memory in Vision-Language-Action Models for Robotic Manipulation26 Aug
  3. 2025OneTwoVLA: A Unified Vision-Language-Action Model with Adaptive Reasoning17 May

Other references · 42

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  3. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  4. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  5. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  6. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  7. 2025Gemma 3 Technical Reportcited by 1,686
  8. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  9. 2021ViViT: A Video Vision Transformercited by 1,610
  10. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,609
  11. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  12. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
30 more
  1. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 721
  2. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  3. 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 643
  4. 2024TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulationcited by 331
  5. 2024VideoMamba: State Space Model for Efficient Video Understandingcited by 325
  6. 2022Expanding Language-Image Pretrained Models for General Video Recognitioncited by 320
  7. 20243D-VLA: A 3D Vision-Language-Action Generative World Modelcited by 320
  8. 2022Behavior Transformers: Cloning k modes with one stonecited by 299
  9. 2024TraceVLA: Visual Trace Prompting Enhances Spatial-Temporal Awareness for Generalist Robotic Policiescited by 298
  10. 2023Video Understanding with Large Language Models: A Surveycited by 203
  11. 2025Real-Time Execution of Action Chunking Flow Policiescited by 199
  12. 2025DexVLA: Vision-Language Model with Plug-In Diffusion Expert for General Robot Controlcited by 191
  13. 2024Behavior Generation with Latent Actionscited by 159
  14. 2025Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Bettercited by 142
  15. 2025A Careful Examination of Large Behavior Models for Multitask Dexterous Manipulationcited by 125
  16. 2025A Comprehensive Survey on Long Context Language Modelingcited by 113
  17. 2024Scaling Cross-Embodied Learning: One Policy for Manipulation, Navigation, Locomotion and Aviationcited by 108
  18. 2017Vision-Based Multi-Task Manipulation for Inexpensive Robots Using End-To-End Learning from Demonstrationcited by 100
  19. 2026A Pragmatic VLA Foundation Modelcited by 97
  20. 2025MemER: Scaling Up Memory for Robot Control via Experience Retrievalcited by 64
  21. 2025Training-Time Action Conditioning for Efficient Real-Time Chunkingcited by 61
  22. 2025SAM2Act: Integrating Visual Foundation Model with A Memory Architecture for Robotic Manipulationcited by 58
  23. 2024From Multimodal LLMs to Generalist Embodied Agents: Methods and Lessonscited by 42
  24. 2025ContextVLA: Vision-Language-Action Model with Amortized Multi-Frame Contextcited by 34
  25. 2025TA-VLA: Elucidating the Design Space of Torque-aware Vision-Language-Action Modelscited by 30
  26. 2025Learning Long-Context Diffusion Policies via Past-Token Predictioncited by 28
  27. 2025CronusVLA: Towards Efficient and Robust Manipulation via Multi-Frame Vision-Language-Action Modelingcited by 24
  28. 2026BPP: Long-Context Robot Imitation Learning by Focusing on Key History Framescited by 17
  29. 2025History-Aware Visuomotor Policy Learning via Point Trackingcited by 10
  30. 2025CycleManip: Enabling Cyclic Task Manipulation via Effective Historical Perception and Understandingcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 55.

55 papers.

Sort:

By area:

  1. 2026π0.7: a Steerable Generalist Robotic Foundation Model with Emergent Capabilitiescited by 127
  2. 2026EventVLA: Event-Driven Visual Evidence Memory for Long-Horizon Vision-Language-Action Policiescited by 18
  3. 2026G0.5: One Autoregressive Stream for Robot Reasoning and Actioncited by 17
  4. 2026RoboTTT: Context Scaling for Robot Policiescited by 13
  5. 2026MemoryWAM: Efficient World Action Modeling with Persistent Memorycited by 12
  6. 2026Goal2Skill: Long-Horizon Manipulation with Adaptive Planning and Reflectioncited by 9
  7. 2026Long-Horizon Manipulation via Trace-Conditioned VLA Planningcited by 9
  8. 2026MemoryVLA++: Temporal Modeling via Memory and Imagination in Vision-Language-Action Modelscited by 9
  9. 2026μVLA: On Recurrent Memory for Partially Observable Manipulation in VLA Modelscited by 7
  10. 2026VPWEM: Non-Markovian Visuomotor Policy with Working and Episodic Memorycited by 6
  11. 2026Hide-and-Seek in Trajectories: Discovering Failure Signals for VLA Runtime Monitoringcited by 6
  12. 2026DAM-VLA: Decoupled Asynchronous Multimodal Vision Language Action modelcited by 6
38 more
  1. 2026DexWorldModel: Causal Latent World Modeling towards Automated Learning of Embodied Taskscited by 5
  2. 2026Embodied-R1.5: Evolving Physical Intelligence via Embodied Foundation Modelscited by 5
  3. 2026Vesta: A Generalist Embodied Reasoning Modelcited by 5
  4. 2026DIM-WAM: World-Action Modeling with Diverse Historical Event Memorycited by 5
  5. 2026OxyGen: Unified KV Cache Management for VLA Inference under Multi-Task Parallelismcited by 4
  6. 2026Cortex: A Bidirectionally Aligned Embodied Agent Framework for Long-horizon Manipulationcited by 4
  7. 2026NativeMEM: Native Memory Compression for Long-Horizon Robotic Manipulationcited by 4
  8. 2026WorldScape Policy 2.0: Empowering Steerable World Action Modeling with Reasoning-Augmented Memory and In-Context Learningcited by 4
  9. 2026Don't Drop the BATON: Long-Horizon Robot Manipulation via Agentic Subtask Exploration and Transition-aware Memorycited by 4
  10. 2026τ0-VLA: a Hierarchical Robot Foundation Model with World-Model-Guided Test-Time Computationcited by 3
  11. 2026ECHO: Continuous Hierarchical Memory for Vision-Language-Action Modelscited by 2
  12. 2026AURA: Action-Gated Memory for Robot Policies at Constant VRAMcited by 2
  13. 2026MemoryVAM: Integrating Memory into Video Action Model for Robot Manipulationcited by 2
  14. 2026FOCA: Future-Oriented Conditioning for Data-Efficient Vision-Language-Action Adaptationcited by 2
  15. 2026FM-VLA: Force-based Memory for Vision-Language-Action Models in Contact-Rich Manipulationcited by 2
  16. 2026SimpleMemVLA: A Simple but Effective Native-Video Memory for Vision-Language-Action Modelscited by 2
  17. 2026Think Like a Pilot: Fine-Grained Long-Horizon UAV Navigationcited by 1
  18. 2026Freeform Preference Learning for Robotic Manipulationcited by 1
  19. 2026PHR-VLA: Planning Horizon Reasoning for Vision-Language-Action Modelscited by 1
  20. 2026World-Action Models for Robot Learning and Control: A Surveycited by 1
  21. 2026Real-Time Execution with Autoregressive Policiescited by 0
  22. 2026Beyond English: Uncovering the Multilingual Gap in Vision-Language-Action Modelscited by 0
  23. 2026RoboMME-Interference: Benchmarking Robot Memory Under Interferencecited by 0
  24. 2026Vision Language Action (VLA) Models for Unmanned Aerial Robotics and Bimanual Manipulation: A Reviewcited by 0
  25. 2026ACE-Data-0: Human-Centric Ambient Capture as Embodied Data Enginecited by 0
  26. 2026UniMem: Unifying Multimodal Memory and Control for Vision-Language-Action Modelscited by 0
  27. 2026PonderPounce: A Pretrained MLLM as an Episode Context Engine for Robot Controlcited by 0
  28. 2026LM-X: Explainable Vision--Language--Action Modeling via Progress, Event, and Uncertainty Predictioncited by 0
  29. 2026StreamPI: Streaming Multimodal Temporal Modeling for Vision-Language-Action Modelscited by 0
  30. 2026TemporalFlow-VLA: Learning Physically Grounded Execution History for Long-Horizon Robot Manipulationcited by 0
  31. 2026Memory as Plans: World-Action Modeling with Memory-Grounded Planningcited by 0
  32. 2026UniMPA: A Unified Memory-Prediction-Action Model via Action-Grounded Transition Modelingcited by 0
  33. 2026StateMem: Single-State Residual Memory with Adaptive Inference for Vision-Language-Action Policiescited by 0
  34. 2026RoboTalk: Learning Multi-Robot Communication and Coordination from Multimodal Demonstrationscited by 0
  35. 2026DRAM: Delta-rule Recurrent Associative Memory for Robot Manipulation Policiescited by 0
  36. 2026ReCAT: Remember, Count, and Time: Structured Recurrent Memory for Robot Manipulationcited by 0
  37. 2026Revision, Not Restart: Revisable Visual Plans for Closed-Loop World-Action Modelscited by 0
  38. 2026Simple Agentic Memory for Generalist Robot Policiescited by 0

50 most cited of 55 All 55 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026T2Mem: Learning Test-Time Memory for Robotics ↗
  2. 2026MemBodied: Recurrent Associative Memory for Vision-Language-Action Models
  3. 2026ECoMEM: Explicit Concept Memory for Memory-Dependent Robot Control ↗
  4. 2026HiMe: Hierarchical Embodied Memory for Long-Horizon Vision-Language-Action Control
  5. 2026RoboMME: Benchmarking and Understanding Memory for Robotic Generalist Policies
  6. 2026MemCtrl: Using MLLMs as Active Memory Controllers on Embodied Agents
  7. 2026Remember what you did?: Learning Behavioral Memories for Partially Observable Object Manipulation
  8. 2026ReMem-VLA: Empowering Vision-Language-Action Model with Memory via Dual-Level Recurrent Queries