mlsanityData to 14 Oct 2026

MolmoAct2: Action Reasoning Models for Real-world Deployment

arXiv 2605.02881 ↗First posted 4 May 2026cs.ROcited by 50

On the map: LLM reasoning → Spatial and embodied reasoning

also in 1 more
Abstract (arXiv)

Vision-Language-Action (VLA) models aim to provide a single generalist controller for robots, but today's systems fall short on the criteria that matter for real-world deployment. Frontier models are closed, open-weight alternatives are tied to expensive hardware, reasoning-augmented policies pay prohibitive latency for their grounding, and fine-tuned success rates remain below the threshold for dependable use. We present MolmoAct2, a fully open action reasoning model built for practical deployment, advancing its predecessor along five axes. We introduce MolmoER, a VLM backbone specialized for spatial and embodied reasoning, trained on a 3.3M-sample corpus with a specialize-then-rehearse recipe. We release three new datasets spanning low-to-medium cost platforms, including MolmoAct2-BimanualYAM, 720 hours of teleoperated bimanual trajectories that constitute the largest open bimanual dataset to date, together with quality-filtered Franka (DROID) and SO100/101 subsets. We provide OpenFAST, an open-weight, open-data action tokenizer trained on millions of trajectories across five embodiments. We redesign the architecture to graft a flow-matching continuous-action expert onto a discrete-token VLM via per-layer KV-cache conditioning. Finally, we propose MolmoThink, an adaptive-depth reasoning variant that re-predicts depth tokens only for scene regions that change between timesteps, retaining geometric grounding at a fraction of prior latency. In the most extensive empirical study of any open VLA to date, spanning 7 simulation and real-world benchmarks, MolmoAct2 outperforms strong baselines including Pi-05, while MolmoER surpasses GPT-5 and Gemini Robotics ER-1.5 across 13 embodied-reasoning benchmarks. We release model weights, training code, and complete training data. Project page: https://allenai.org/blog/molmoact2

Key facts. Cited by 50 papers in the map (data through 14 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 62 of its 94 references that are in the map. First posted on arXiv 4 May 2026.

Builds on

62 of 94 references.

Closest ancestors · 10

Same direction, newest first

  1. 2026Recurrent-Depth VLA: Implicit Test-Time Compute Scaling of Vision-Language-Action Models via Latent Iterative Reasoning8 Feb
  2. 2025SIMS-V: Simulated Instruction-Tuning for Spatial Video Understanding6 Nov
  3. 2025MolmoAct: Action Reasoning Models that can Reason in Space11 Aug
  4. 2025ThinkAct: Vision-Language-Action Reasoning via Reinforced Visual Latent Planning22 Jul
  5. 2025RoboRefer: Towards Spatial Referring with Reasoning in Vision-Language Models for Robotics4 Jun
  6. 2025NORA: A Small Open-Sourced Generalist Vision Language Action Model for Embodied Tasks28 Apr
  7. 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success27 Feb
  8. 2024Emma-X: An Embodied Multimodal Action Model with Grounded Chain of Thought and Look-ahead Spatial Reasoning16 Dec
  9. 2024Robotic Control via Embodied Chain-of-Thought Reasoning11 Jul
  10. 2023RoboVQA: Multimodal Long-Horizon Reasoning for Robotics1 Nov

Other references · 52

  1. 2025Qwen3 Technical Reportcited by 8,820
  2. 2022Flow Matching for Generative Modelingcited by 5,702
  3. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  4. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  5. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  6. 2025Qwen3-VL Technical Reportcited by 2,793
  7. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  8. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  9. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  10. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,761
  11. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
  12. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
38 more
  1. 2024Depth Anything V2cited by 1,422
  2. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
  3. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 1,053
  4. 2025OpenAI GPT-5 System Cardcited by 989
  5. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 852
  6. 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 691
  7. 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 665
  8. 2025SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Roboticscited by 530
  9. 2025SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Modelcited by 519
  10. 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 490
  11. 2024CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulationcited by 422
  12. 2024TraceVLA: Visual Trace Prompting Enhances Spatial-Temporal Awareness for Generalist Robotic Policiescited by 304
  13. 2025X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Modelcited by 303
  14. 2026Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planningcited by 300
  15. 2019RoboNet: Large-Scale Multi-Robot Learningcited by 270
  16. 2024RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Roboticscited by 251
  17. 2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 250
  18. 2025Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasetscited by 240
  19. 2025DexVLA: Vision-Language Model with Plug-In Diffusion Expert for General Robot Controlcited by 195
  20. 2023RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shotcited by 183
  21. 2025Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Bettercited by 146
  22. 2026Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Groundingcited by 139
  23. 2024ALOHA Unleashed: A Simple Recipe for Robot Dexteritycited by 134
  24. 2025Cambrian-S: Towards Spatial Supersensing in Videocited by 134
  25. 2025HAMSTER: Hierarchical Action Models For Open-World Robot Manipulationcited by 130
  26. 2024SAT: Dynamic Spatial Aptitude Training for Multimodal Language Modelscited by 114
  27. 2023On Bringing Robots Homecited by 101
  28. 2025Visual Spatial Tuningcited by 89
  29. 2025Gemini Robotics 1.5: Pushing the Frontier of Generalist Robots with Advanced Embodied Reasoning, Thinking, and Motion Transfercited by 87
  30. 2025Gemini Robotics: Bringing AI into the Physical Worldcited by 83
  31. 2025Scaling Spatial Intelligence with Multimodal Foundation Modelscited by 82
  32. 2023All in Tokens: Unifying Output Space of Visual Tasks via Soft Tokencited by 52
  33. 2025SARM: Stage-Aware Reward Modeling for Long Horizon Robot Manipulationcited by 42
  34. 2025RoboEval: Where Robotic Manipulation Meets Structured and Scalable Evaluationcited by 32
  35. 2025PointArena: Probing Multimodal Grounding Through Language-Guided Pointingcited by 31
  36. 2025StereoVLA: Enhancing Vision-Language-Action Models with Stereo Visioncited by 18
  37. 2026LAP: Language-Action Pre-Training Enables Zero-shot Cross-Embodiment Transfercited by 18
  38. 2026MolmoSpaces: A Large-Scale Open Ecosystem for Robot Navigation and Manipulationcited by 14

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 50.

50 papers.

Sort:

By area:

  1. 2026Qwen-RobotManip Technical Report: Alignment Unlocks Scale for Robotic Manipulation Foundation Modelscited by 50
  2. 2026RoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policiescited by 40
  3. 2026Xiaomi-Robotics-1: Scaling Vision-Language-Action Models with over 100K Hours of Real-World Trajectoriescited by 29
  4. 2026RoboTTT: Context Scaling for Robot Policiescited by 19
  5. 2026Flex-π: A Multi-Stream World-Action Model with Compute Flexibilitycited by 11
  6. 2026SafeVLA-Bench: A Benchmark for the Success-Safety Gap in Vision-Language-Action Modelscited by 9
  7. 2026Training Vision-Language-Action Models with Dense Embodied Chain-of-Thought Supervisioncited by 8
  8. 2026Embodied-R1.5: Evolving Physical Intelligence via Embodied Foundation Modelscited by 6
  9. 2026LA4VLA: Learning to Act without Seeing via Language-Action Pretrainingcited by 6
  10. 2026Data Pyramid for Embodied Manipulation: A Surveycited by 6
  11. 2026MolmoMotion: Forecasting Point Trajectories in 3D with Language Instructioncited by 5
  12. 2026ACE-Brain-0.5: A Unified Embodied Foundational Model for Physical Agentic AIcited by 5
38 more
  1. 2026Uncertainty Quantification for Flow-Based Vision-Language-Action Modelscited by 4
  2. 2026Coarse-to-Control: Action-Token Planning for Vision-Language-Action Modelscited by 3
  3. 2026HumanScale: Egocentric Human Video Can Outperform Real-Robot Data for Embodied Pretrainingcited by 3
  4. 2026Understanding and Mitigating the Video-Action Generalization Gap via Temporal Ratiocited by 3
  5. 2026LIRA: Local Cross-Layer Information Routing for Vision-Language-Action Decodingcited by 3
  6. 2026Hydra-0: Action Flow for Generalist World Modeling and Controlcited by 3
  7. 2026Sequential Planning via Anchored Robotic Keypointscited by 2
  8. 2026RxBrain: Embodied Cognition Foundation Model with Joint Language-Visual Reasoning and Imaginationcited by 2
  9. 2026You Don't Need To Stay in The Loop: An Agentic Robotics Loop for Robot-Policy Improvementcited by 2
  10. 2026DREAM: Deployment-Time Demonstration Generation via Real-to-Sim for Scalable Policy Adaptationcited by 2
  11. 2026VLA Grounder: Language-Conditioning Space Optimization for Black-Box VLA Modelscited by 1
  12. 2026DiMaS: Distribution Matching for Steering Vision-Language-Action Modelscited by 1
  13. 2026ArmnetBench v0.1: Parallel Real-World Evaluation of Manipulation Policies on a Low-Cost Arm Farmcited by 1
  14. 2026CLAP: Cross-Embodiment Video World Models are Zero-Shot Physical Simulatorscited by 1
  15. 2026PHR-VLA: Planning Horizon Reasoning for Vision-Language-Action Modelscited by 1
  16. 2026LightNav-0: Eliciting VLM Spatial Intelligence for Generalist Embodied Navigationcited by 1
  17. 2026SkipVLA: Skipping VLA Steps with Classical Planning for Fast Robot Manipulationcited by 1
  18. 2026InternW0-Δ: A World Action Model Bridging Predictive Dynamics and Actions with 20K+ Hours of Open Datacited by 1
  19. 2026Towards Trustworthy Physical Intelligence: From Theory to Practice Across Life Cyclecited by 0
  20. 2026From Recovery to Drop-off: How Action Post-training Reduces a VLM's Late-Layer Depth Decodabilitycited by 0
  21. 2026LIBERO-VIFO: Benchmarking the Capability and Safety of Visual Cue Following in Vision-Language-Action Modelscited by 0
  22. 2026R3: Training Robots to Reason in Natural Language via Reinforcement Learningcited by 0
  23. 2026One Policy, Many Embodiments: Unified Camera-Centric Action Geometry Pre-training for Heterogeneous Embodied Manipulationcited by 0
  24. 2026H-VLA: Hierarchical Vision-Language-Action Model with Key-Action Reasoning and Motion Planning in a Unified Action Spacecited by 0
  25. 2026Grounded Action Model: 3D Grounding as a Foundation for Roboticscited by 0
  26. 2026Adjoint Guidance Flow: Amortized Critic Guidance for VLA Policiescited by 0
  27. 2026FineART: Fine-Grained Annotated Robotic Trajectory Dataset and Vision-Language-Action Model for Bimanual Manipulationcited by 0
  28. 2026LIBERO-MAX: Do Robot Policies Adapt When the World Changes?cited by 0
  29. 2026MotorMind: Scaffolding General Vision Language Models for Zero-Shot Robot Manipulationcited by 0
  30. 2026Rho: A Foundation for Efficiently Adaptable VLA Modelscited by 0
  31. 2026BIND: Binding 3D Robot Actions to 2D Image Featurescited by 0
  32. 2026RoboCoach: World Models as Active Coaches for Compositional Robot Skillscited by 0
  33. 2026FastOPD: On-Policy Distillation for Lightweight VLA Deploymentcited by 0
  34. 2026RoboAware: Learning to Coordinate Embodied Skills from Counterfactual Outcomescited by 0
  35. 2026REACT: Rolling Denoising and Dual Decoupling for Reactive Robot Control with VLA Modelscited by 0
  36. 2026StereoEngine: Scaling Stereo Bimanual Robot Data without Stereo Camerascited by 0
  37. 2026S-Space: Exploring Spatial Workspace in Multimodal Language Modelscited by 0
  38. 2026RAIN: Robotic Task Generalization with Region-Aware Interaction Networkscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026G0.5: One Autoregressive Stream for Robot Reasoning and Action
  2. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applications
  3. 2025VLA-0: Building State-of-the-Art VLAs with Zero Modification
  4. 2025An Anatomy of Vision-Language-Action Models: From Modules to Milestones and Challenges
  5. 2025ChatVLA-2: Vision-Language-Action Model with Open-World Embodied Reasoning from Pretrained Knowledge
  6. 2026Does VLA Even Know the Basics? Measuring Commonsense and World Knowledge Retention in Vision-Language-Action Models
  7. 2025Unified Embodied VLM Reasoning with Robotic Action via Autoregressive Discretized Pre-training
  8. 2024What Matters in Building Vision-Language-Action Models for Generalist Robots