mlsanityData to 14 Oct 2026

Robix: A Unified Model for Robot Interaction, Reasoning and Planning

arXiv 2509.01106 ↗First posted 1 Sep 2025cs.AI cs.CV cs.ROcited by 23

Not on the map. Why?

Abstract (arXiv)

We introduce Robix, a unified model that integrates robot reasoning, task planning, and natural language interaction within a single vision-language architecture. Acting as the high-level cognitive layer in a hierarchical robot system, Robix dynamically generates atomic commands for the low-level controller and verbal responses for human interaction, enabling robots to follow complex instructions, plan long-horizon tasks, and interact naturally with human within an end-to-end framework. Robix further introduces novel capabilities such as proactive dialogue, real-time interruption handling, and context-aware commonsense reasoning during task execution. At its core, Robix leverages chain-of-thought reasoning and adopts a three-stage training strategy: (1) continued pretraining to enhance foundational embodied reasoning abilities including 3D spatial understanding, visual grounding, and task-centric reasoning; (2) supervised finetuning to model human-robot interaction and task planning as a unified reasoning-action sequence; and (3) reinforcement learning to improve reasoning-action consistency and long-horizon task coherence. Extensive experiments demonstrate that Robix outperforms both open-source and commercial baselines (e.g., GPT-4o and Gemini 2.5 Pro) in interactive task execution, demonstrating strong generalization across diverse instruction types (e.g., open-ended, multi-stage, constrained, invalid, and interrupted) and various user-involved tasks such as table bussing, grocery shopping, and dietary filtering.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Builds on 73 of its 86 references that are in the map. First posted on arXiv 1 Sep 2025.

Builds on

73 of 86 references.

References · 73

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  3. 2025Qwen2.5-VL Technical Reportcited by 5,587
  4. 2024Qwen2.5 Technical Reportcited by 4,942
  5. 2024GPT-4o System Cardcited by 4,588
  6. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  7. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  8. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  9. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  10. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  11. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  12. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
38 more
  1. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  2. 2024Depth Anything V2cited by 1,422
  3. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
  4. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,217
  5. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,081
  6. 2016Modeling Context in Referring Expressionscited by 1,063
  7. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 852
  8. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
  9. 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 764
  10. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 744
  11. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 680
  12. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 657
  13. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 611
  14. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 610
  15. 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 550
  16. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Datasetcited by 526
  17. 2022LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Modelscited by 462
  18. 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 454
  19. 2025AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systemscited by 405
  20. 2024Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robotscited by 394
  21. 2025Seed1.5-VL Technical Reportcited by 373
  22. 2024Robotic Control via Embodied Chain-of-Thought Reasoningcited by 346
  23. 2022Visual Spatial Reasoningcited by 293
  24. 2024RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Roboticscited by 251
  25. 2025Hi Robot: Open-Ended Instruction Following with Hierarchical Vision-Language-Action Modelscited by 221
  26. 2024NaVILA: Legged Robot Vision-Language-Action Model for Navigationcited by 211
  27. 2025EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Videocited by 211
  28. 2024RT-H: Action Hierarchies Using Languagecited by 210
  29. 2019RIO: 3D Object Instance Re-Localization in Changing Indoor Environmentscited by 188
  30. 2023HoloAssist: an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real Worldcited by 166
  31. 2025Cosmos-Reason1: From Physical Common Sense To Embodied Reasoningcited by 164
  32. 2023Look Before You Leap: Unveiling the Power of GPT-4V in Robotic Vision-Language Planningcited by 161
  33. 2024Yell At Your Robot: Improving On-the-Fly from Language Correctionscited by 124
  34. 2023RoboVQA: Multimodal Long-Horizon Reasoning for Roboticscited by 120
  35. 2022EgoTaskQA: Understanding Human Tasks in Egocentric Videoscited by 115
  36. 2024SAT: Dynamic Spatial Aptitude Training for Multimodal Language Modelscited by 114
  37. 2025GR-3 Technical Reportcited by 111
  38. 2024EmbSpatial-Bench: Benchmarking Spatial Understanding for Embodied Tasks with Large Vision-Language Modelscited by 107

50 most cited of 73 All 73 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 17.

23 papers.

Sort:

By area:

  1. 2025Actions as Language: Fine-Tuning VLMs into VLAs Without Catastrophic Forgettingcited by 58
  2. 2025Vlaser: Vision-Language-Action Model with Synergistic Embodied Reasoningcited by 23
  3. 2026Steerable Vision-Language-Action Policies for Embodied Reasoning and Hierarchical Controlcited by 20
  4. 2026AURA: Always-On Understanding and Real-Time Assistance via Video Streamscited by 16
  5. 2025SpatialDreamer: Incentivizing Spatial Reasoning via Active Mental Imagerycited by 11
  6. 2026Demystifying Action Space Design for Robotic Manipulation Policiescited by 9
  7. 2026Long-Horizon Manipulation via Trace-Conditioned VLA Planningcited by 9
  8. 2025Mind to Hand: Purposeful Robotic Control via Embodied Reasoningcited by 8
  9. 2026ForeAct: Steering Your VLA with Efficient Visual Foresight Planningcited by 8
  10. 2025Abstract 3D Perception for Spatial Intelligence in Vision-Language Modelscited by 7
  11. 2026RoboAgent: Chaining Basic Capabilities for Embodied Task Planningcited by 7
  12. 2025How Do VLAs Effectively Inherit from VLMs?cited by 6
11 more
  1. 2026VLA-Thinker: Boosting Vision-Language-Action Models through Thinking-with-Image Reasoningcited by 6
  2. 2026MindVLA-U1: VLA Beats VA with Unified Streaming Architecture for Autonomous Drivingcited by 6
  3. 2026KEEP: A KV-Cache-Centric Memory Management System for Efficient Embodied Planningcited by 5
  4. 2026ReMoT: Reinforcement Learning with Motion Contrast Tripletscited by 3
  5. 2026StreamArena: Toward Continuous, Interactive, and Long-Horizon Agentic Streaming Video Understandingcited by 3
  6. 2026Lucida: Parse, Generate, and Place for Composable Real-to-Sim Scene Modelingcited by 2
  7. 2026RoboEvolve: Co-Evolving Planner-Simulator for Robotic Manipulation with Limited Datacited by 1
  8. 2026Task-Focused Memorization for Multimodal Agentscited by 1
  9. 2026RoboInter1.5: A Holistic Intermediate Representation Suite for Embodied World Modeling and Robotic Manipulationcited by 0
  10. 2026GST-Bench: Can VLMs Develop Global Spatial Awareness from Video?cited by 0
  11. 2026EmbodiedMind: Adaptive Data Curation and Prefix-Tree Reinforcement Learning for Efficient Embodied Intelligencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025RoBridge: A Hierarchical Architecture Bridging Cognition and Execution for General Robotic Manipulation
  2. 2025Unified Embodied VLM Reasoning with Robotic Action via Autoregressive Discretized Pre-training
  3. 2026WCM: World-Cognition Model for Generalizable Human-Robot Interaction
  4. 2025Robobench: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models as Embodied Brain
  5. 2025Robot-R1: Reinforcement Learning for Enhanced Embodied Reasoning in Robotics
  6. 2026One Word, Different Action: A Real-Robot Benchmark for Language-Conditioned Embodied Reasoning ↗
  7. 2024RoboMatrix: A Skill-centric Hierarchical Framework for Scalable Robot Task Planning and Execution in Open-World
  8. 2025Vision-Language-Policy Model for Dynamic Robot Task Planning