mlsanityData to 7 Oct 2026

SAM2Act: Integrating Visual Foundation Model with A Memory Architecture for Robotic Manipulation

arXiv 2501.18564 ↗First posted 30 Jan 2025cs.ROcited by 58

Not on the map. Why?

Cited by 27 in Vision-language-action models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Robotic manipulation systems operating in diverse, dynamic environments must exhibit three critical abilities: multitask interaction, generalization to unseen scenarios, and spatial memory. While significant progress has been made in robotic manipulation, existing approaches often fall short in generalization to complex environmental variations and addressing memory-dependent tasks. To bridge this gap, we introduce SAM2Act, a multi-view robotic transformer-based policy that leverages multi-resolution upsampling with visual representations from large-scale foundation model. SAM2Act achieves a state-of-the-art average success rate of 86.8% across 18 tasks in the RLBench benchmark, and demonstrates robust generalization on The Colosseum benchmark, with only a 4.3% performance gap under diverse environmental perturbations. Building on this foundation, we propose SAM2Act+, a memory-based architecture inspired by SAM2, which incorporates a memory bank, an encoder, and an attention mechanism to enhance spatial memory. To address the need for evaluating memory-dependent tasks, we introduce MemoryBench, a novel benchmark designed to assess spatial memory and action recall in robotic manipulation. SAM2Act+ achieves an average success rate of 94.3% on memory-based tasks in MemoryBench, significantly outperforming existing approaches and pushing the boundaries of memory-based robotic systems. Project page: sam2act.github.io.

Key facts. Cited by 58 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 Jan 2025.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 18; 2026: 40.

58 papers.

Sort:

By area:

  1. 2026π0.7: a Steerable Generalist Robotic Foundation Model with Emergent Capabilitiescited by 127
  2. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 82
  3. 2025MemER: Scaling Up Memory for Robot Control via Experience Retrievalcited by 64
  4. 2026MEM: Multi-Scale Embodied Memory for Vision Language Action Modelscited by 55
  5. 2026RMBench: Memory-Dependent Robotic Manipulation Benchmark with Insights into Policy Designcited by 40
  6. 2025Vision Language Action Models in Robotic Manipulation: A Systematic Reviewcited by 38
  7. 2025Memory, Benchmark & Robots: A Benchmark for Solving Complex Tasks with Reinforcement Learningcited by 34
  8. 2025Exploring the Limits of Vision-Language-Action Manipulations in Cross-task Generalizationcited by 31
  9. 2026ReMem-VLA: Empowering Vision-Language-Action Model with Memory via Dual-Level Recurrent Queriescited by 28
  10. 2026RLDX-1 Technical Reportcited by 25
  11. 2025Rethinking Progression of Memory State in Robotic Manipulation: An Object-Centric Perspectivecited by 20
  12. 2026EventVLA: Event-Driven Visual Evidence Memory for Long-Horizon Vision-Language-Action Policiescited by 18
38 more
  1. 2025Seeing to Act, Prompting to Specify: A Bayesian Factorization of Vision Language Action Policycited by 17
  2. 2026Gated Memory Policy: In-Context Memorization and Adaptationcited by 13
  3. 2026Wall-OSS-0.5 Technical Reportcited by 11
  4. 2026MemoryVLA++: Temporal Modeling via Memory and Imagination in Vision-Language-Action Modelscited by 9
  5. 2026HiMem-WAM: Hierarchical Memory-Gated World Action Models for Robotic Manipulationcited by 8
  6. 2025One Trajectory, One Token: Grounded Video Tokenization via Panoptic Sub-object Trajectorycited by 7
  7. 2026Non-Markovian Long-Horizon Robot Manipulation via Keyframe Chainingcited by 7
  8. 2026Gaze-Regularized Vision-Language-Action Models for Robotic Manipulationcited by 7
  9. 2025Focusing on What Matters: Object-Agent-centric Tokenization for Vision Language Action modelscited by 6
  10. 2025ELMUR: External Layer Memory with Update/Rewrite for Long-Horizon RL Problemscited by 6
  11. 2026RoboStream: Weaving Spatio-Temporal Reasoning with Memory in Vision-Language Models for Roboticscited by 6
  12. 2025SpatialActor: Exploring Disentangled Spatial Representations for Robust Robotic Manipulationcited by 5
  13. 2026KEMO: Event-Driven Keyframe Memory for Long-Horizon Robot Manipulation with VLA Policiescited by 5
  14. 2026DIM-WAM: World-Action Modeling with Diverse Historical Event Memorycited by 5
  15. 2026Chameleon: Control-Indexed Prospective Memory for Visuomotor Manipulationcited by 4
  16. 2026NativeMEM: Native Memory Compression for Long-Horizon Robotic Manipulationcited by 4
  17. 2026Data Pyramid for Embodied Manipulation: A Surveycited by 4
  18. 2025Robotic Manipulation via Imitation Learning: Taxonomy, Evolution, Benchmark, and Challengescited by 3
  19. 2026Notes-to-Self: Scratchpad Augmented VLAs for Memory Dependent Manipulation Taskscited by 3
  20. 2025Towards Fast, Memory-based and Data-Efficient Vision-Language Policycited by 2
  21. 2025Generalizable Hierarchical Skill Learning via Object-Centric Representationcited by 2
  22. 2026PhysMem: Scaling Test-Time Memory for Embodied Physical Reasoningcited by 2
  23. 2026SimpleMemVLA: A Simple but Effective Native-Video Memory for Vision-Language-Action Modelscited by 2
  24. 2025SAM2Grasp: Resolve Multi-modal Grasping via Prompt-conditioned Temporal Action Predictioncited by 1
  25. 2026Memory Retention Is Not Enough to Master Memory Tasks in Reinforcement Learningcited by 1
  26. 2026Boosting SAM for Cross-Domain Few-Shot Segmentation via Conditional Point Sparsificationcited by 1
  27. 2026VLM-DEWM: Dynamic External World Model for Verifiable and Resilient Vision-Language Planning in Manufacturingcited by 1
  28. 2026WatchAct: A Benchmark for Behavior-Grounded Robot Manipulationcited by 1
  29. 2026HiMe: Hierarchical Embodied Memory for Long-Horizon Vision-Language-Action Controlcited by 1
  30. 2026BridgeVLA++: A Data-Efficient, Generalizable, and Memory-Augmented Vision-Language-Action Framework for 3D Manipulationcited by 1
  31. 2025Perceiving, Reasoning, Adapting: A Dual-Layer Framework for VLM-Guided Precision Robotic Manipulationcited by 0
  32. 2025Mini Diffuser: Fast Multi-task Diffusion Policy Training Using Two-level Mini-batchescited by 0
  33. 2025RoboPearls: Editable Video Simulation for Robot Manipulationcited by 0
  34. 2026Decompose and Recompose: Reasoning New Skills from Existing Abilities for Cross-Task Robotic Manipulationcited by 0
  35. 2026Differential Amplifier-Inspired AmpAttention for Multi-View Robotic Manipulationcited by 0
  36. 2026UniMem: Unifying Multimodal Memory and Control for Vision-Language-Action Modelscited by 0
  37. 2026PonderPounce: A Pretrained MLLM as an Episode Context Engine for Robot Controlcited by 0
  38. 2026AGM: Achievement-Grounded Memory for Closed-Loop Agents with Frozen VLA Policiescited by 0

50 most cited of 58 All 58 (JSON)