Robix: A Unified Model for Robot Interaction, Reasoning and Planning
Not on the map. Why?
Abstract (arXiv)
We introduce Robix, a unified model that integrates robot reasoning, task planning, and natural language interaction within a single vision-language architecture. Acting as the high-level cognitive layer in a hierarchical robot system, Robix dynamically generates atomic commands for the low-level controller and verbal responses for human interaction, enabling robots to follow complex instructions, plan long-horizon tasks, and interact naturally with human within an end-to-end framework. Robix further introduces novel capabilities such as proactive dialogue, real-time interruption handling, and context-aware commonsense reasoning during task execution. At its core, Robix leverages chain-of-thought reasoning and adopts a three-stage training strategy: (1) continued pretraining to enhance foundational embodied reasoning abilities including 3D spatial understanding, visual grounding, and task-centric reasoning; (2) supervised finetuning to model human-robot interaction and task planning as a unified reasoning-action sequence; and (3) reinforcement learning to improve reasoning-action consistency and long-horizon task coherence. Extensive experiments demonstrate that Robix outperforms both open-source and commercial baselines (e.g., GPT-4o and Gemini 2.5 Pro) in interactive task execution, demonstrating strong generalization across diverse instruction types (e.g., open-ended, multi-stage, constrained, invalid, and interrupted) and various user-involved tasks such as table bussing, grocery shopping, and dietary filtering.
Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Builds on 73 of its 86 references that are in the map. First posted on arXiv 1 Sep 2025.
Builds on
73 of 86 references.
References · 73
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024Qwen2.5 Technical Reportcited by 4,942
- 2024GPT-4o System Cardcited by 4,588
- 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
- 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
- 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
- 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
- 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
- 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
38 more
- 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
- 2024Depth Anything V2cited by 1,422
- 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
- 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,217
- 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,081
- 2016Modeling Context in Referring Expressionscited by 1,063
- 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 852
- 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
- 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 764
- 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 744
- 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 680
- 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 657
- 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 611
- 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 610
- 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 550
- 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Datasetcited by 526
- 2022LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Modelscited by 462
- 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 454
- 2025AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systemscited by 405
- 2024Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robotscited by 394
- 2025Seed1.5-VL Technical Reportcited by 373
- 2024Robotic Control via Embodied Chain-of-Thought Reasoningcited by 346
- 2022Visual Spatial Reasoningcited by 293
- 2024RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Roboticscited by 251
- 2025Hi Robot: Open-Ended Instruction Following with Hierarchical Vision-Language-Action Modelscited by 221
- 2024NaVILA: Legged Robot Vision-Language-Action Model for Navigationcited by 211
- 2025EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Videocited by 211
- 2024RT-H: Action Hierarchies Using Languagecited by 210
- 2019RIO: 3D Object Instance Re-Localization in Changing Indoor Environmentscited by 188
- 2023HoloAssist: an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real Worldcited by 166
- 2025Cosmos-Reason1: From Physical Common Sense To Embodied Reasoningcited by 164
- 2023Look Before You Leap: Unveiling the Power of GPT-4V in Robotic Vision-Language Planningcited by 161
- 2024Yell At Your Robot: Improving On-the-Fly from Language Correctionscited by 124
- 2023RoboVQA: Multimodal Long-Horizon Reasoning for Roboticscited by 120
- 2022EgoTaskQA: Understanding Human Tasks in Egocentric Videoscited by 115
- 2024SAT: Dynamic Spatial Aptitude Training for Multimodal Language Modelscited by 114
- 2025GR-3 Technical Reportcited by 111
- 2024EmbSpatial-Bench: Benchmarking Spatial Understanding for Embodied Tasks with Large Vision-Language Modelscited by 107
50 most cited of 73 All 73 (JSON)
Built on by
Citing papers per year: 2025: 6; 2026: 17.
23 papers.
Sort:
By area:
- 2025Actions as Language: Fine-Tuning VLMs into VLAs Without Catastrophic Forgettingcited by 58
- 2025Vlaser: Vision-Language-Action Model with Synergistic Embodied Reasoningcited by 23
- 2026Steerable Vision-Language-Action Policies for Embodied Reasoning and Hierarchical Controlcited by 20
- 2026AURA: Always-On Understanding and Real-Time Assistance via Video Streamscited by 16
- 2025SpatialDreamer: Incentivizing Spatial Reasoning via Active Mental Imagerycited by 11
- 2026Demystifying Action Space Design for Robotic Manipulation Policiescited by 9
- 2026Long-Horizon Manipulation via Trace-Conditioned VLA Planningcited by 9
- 2025Mind to Hand: Purposeful Robotic Control via Embodied Reasoningcited by 8
- 2026ForeAct: Steering Your VLA with Efficient Visual Foresight Planningcited by 8
- 2025Abstract 3D Perception for Spatial Intelligence in Vision-Language Modelscited by 7
- 2026RoboAgent: Chaining Basic Capabilities for Embodied Task Planningcited by 7
- 2025How Do VLAs Effectively Inherit from VLMs?cited by 6
11 more
- 2026VLA-Thinker: Boosting Vision-Language-Action Models through Thinking-with-Image Reasoningcited by 6
- 2026MindVLA-U1: VLA Beats VA with Unified Streaming Architecture for Autonomous Drivingcited by 6
- 2026KEEP: A KV-Cache-Centric Memory Management System for Efficient Embodied Planningcited by 5
- 2026ReMoT: Reinforcement Learning with Motion Contrast Tripletscited by 3
- 2026StreamArena: Toward Continuous, Interactive, and Long-Horizon Agentic Streaming Video Understandingcited by 3
- 2026Lucida: Parse, Generate, and Place for Composable Real-to-Sim Scene Modelingcited by 2
- 2026RoboEvolve: Co-Evolving Planner-Simulator for Robotic Manipulation with Limited Datacited by 1
- 2026Task-Focused Memorization for Multimodal Agentscited by 1
- 2026RoboInter1.5: A Holistic Intermediate Representation Suite for Embodied World Modeling and Robotic Manipulationcited by 0
- 2026GST-Bench: Can VLMs Develop Global Spatial Awareness from Video?cited by 0
- 2026EmbodiedMind: Adaptive Data Curation and Prefix-Tree Reinforcement Learning for Efficient Embodied Intelligencecited by 0
- 2026EmbodiedMind: Adaptive Data Curation and Prefix-Tree Reinforcement Learning for Efficient Embodied Intelligence17 Sep 2026
- 2026Lucida: Parse, Generate, and Place for Composable Real-to-Sim Scene Modeling31 Aug 2026
- 2026GST-Bench: Can VLMs Develop Global Spatial Awareness from Video?6 Aug 2026
- 2026StreamArena: Toward Continuous, Interactive, and Long-Horizon Agentic Streaming Video Understanding6 Aug 2026
- 2026RoboInter1.5: A Holistic Intermediate Representation Suite for Embodied World Modeling and Robotic Manipulation21 Jul 2026
- 2026Task-Focused Memorization for Multimodal Agents29 May 2026
- 2026RoboEvolve: Co-Evolving Planner-Simulator for Robotic Manipulation with Limited Data13 May 2026
- 2026MindVLA-U1: VLA Beats VA with Unified Streaming Architecture for Autonomous Driving12 May 2026
- 2026Long-Horizon Manipulation via Trace-Conditioned VLA Planning23 Apr 2026
- 2026RoboAgent: Chaining Basic Capabilities for Embodied Task Planning9 Apr 2026
- 2026AURA: Always-On Understanding and Real-Time Assistance via Video Streams5 Apr 2026
- 2026VLA-Thinker: Boosting Vision-Language-Action Models through Thinking-with-Image Reasoning15 Mar 2026
11 more
- 2026ReMoT: Reinforcement Learning with Motion Contrast Triplets28 Feb 2026
- 2026KEEP: A KV-Cache-Centric Memory Management System for Efficient Embodied Planning27 Feb 2026
- 2026Demystifying Action Space Design for Robotic Manipulation Policies26 Feb 2026
- 2026Steerable Vision-Language-Action Policies for Embodied Reasoning and Hierarchical Control13 Feb 2026
- 2026ForeAct: Steering Your VLA with Efficient Visual Foresight Planning12 Feb 2026
- 2025Mind to Hand: Purposeful Robotic Control via Embodied Reasoning9 Dec 2025
- 2025SpatialDreamer: Incentivizing Spatial Reasoning via Active Mental Imagery8 Dec 2025
- 2025Abstract 3D Perception for Spatial Intelligence in Vision-Language Models14 Nov 2025
- 2025How Do VLAs Effectively Inherit from VLMs?10 Nov 2025
- 2025Vlaser: Vision-Language-Action Model with Synergistic Embodied Reasoning13 Oct 2025
- 2025Actions as Language: Fine-Tuning VLMs into VLAs Without Catastrophic Forgetting26 Sep 2025
- 2025Actions as Language: Fine-Tuning VLMs into VLAs Without Catastrophic Forgettingcited by 58
- 2025Vlaser: Vision-Language-Action Model with Synergistic Embodied Reasoningcited by 23
- 2026Steerable Vision-Language-Action Policies for Embodied Reasoning and Hierarchical Controlcited by 20
- 2025SpatialDreamer: Incentivizing Spatial Reasoning via Active Mental Imagerycited by 11
- 2025Mind to Hand: Purposeful Robotic Control via Embodied Reasoningcited by 8
- 2026VLA-Thinker: Boosting Vision-Language-Action Models through Thinking-with-Image Reasoningcited by 6
- 2026VLA-Thinker: Boosting Vision-Language-Action Models through Thinking-with-Image Reasoning15 Mar 2026
- 2026Steerable Vision-Language-Action Policies for Embodied Reasoning and Hierarchical Control13 Feb 2026
- 2025Mind to Hand: Purposeful Robotic Control via Embodied Reasoning9 Dec 2025
- 2025SpatialDreamer: Incentivizing Spatial Reasoning via Active Mental Imagery8 Dec 2025
- 2025Vlaser: Vision-Language-Action Model with Synergistic Embodied Reasoning13 Oct 2025
- 2025Actions as Language: Fine-Tuning VLMs into VLAs Without Catastrophic Forgetting26 Sep 2025
- 2026Long-Horizon Manipulation via Trace-Conditioned VLA Planningcited by 9
- 2026ForeAct: Steering Your VLA with Efficient Visual Foresight Planningcited by 8
- 2025How Do VLAs Effectively Inherit from VLMs?cited by 6
- 2026MindVLA-U1: VLA Beats VA with Unified Streaming Architecture for Autonomous Drivingcited by 6
- 2026MindVLA-U1: VLA Beats VA with Unified Streaming Architecture for Autonomous Driving12 May 2026
- 2026Long-Horizon Manipulation via Trace-Conditioned VLA Planning23 Apr 2026
- 2026ForeAct: Steering Your VLA with Efficient Visual Foresight Planning12 Feb 2026
- 2025How Do VLAs Effectively Inherit from VLMs?10 Nov 2025
- 2026AURA: Always-On Understanding and Real-Time Assistance via Video Streamscited by 16
- 2026Demystifying Action Space Design for Robotic Manipulation Policiescited by 9
- 2026RoboAgent: Chaining Basic Capabilities for Embodied Task Planningcited by 7
- 2026Lucida: Parse, Generate, and Place for Composable Real-to-Sim Scene Modelingcited by 2
- 2026RoboEvolve: Co-Evolving Planner-Simulator for Robotic Manipulation with Limited Datacited by 1
- 2026GST-Bench: Can VLMs Develop Global Spatial Awareness from Video?cited by 0
- 2026Lucida: Parse, Generate, and Place for Composable Real-to-Sim Scene Modeling31 Aug 2026
- 2026GST-Bench: Can VLMs Develop Global Spatial Awareness from Video?6 Aug 2026
- 2026RoboEvolve: Co-Evolving Planner-Simulator for Robotic Manipulation with Limited Data13 May 2026
- 2026RoboAgent: Chaining Basic Capabilities for Embodied Task Planning9 Apr 2026
- 2026AURA: Always-On Understanding and Real-Time Assistance via Video Streams5 Apr 2026
- 2026Demystifying Action Space Design for Robotic Manipulation Policies26 Feb 2026
Similar, not cited
Close in wording, no citation either way
- 2025RoBridge: A Hierarchical Architecture Bridging Cognition and Execution for General Robotic Manipulation
- 2025Unified Embodied VLM Reasoning with Robotic Action via Autoregressive Discretized Pre-training
- 2026WCM: World-Cognition Model for Generalizable Human-Robot Interaction
- 2025Robobench: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models as Embodied Brain
- 2025Robot-R1: Reinforcement Learning for Enhanced Embodied Reasoning in Robotics
- 2026One Word, Different Action: A Real-Robot Benchmark for Language-Conditioned Embodied Reasoning ↗
- 2024RoboMatrix: A Skill-centric Hierarchical Framework for Scalable Robot Task Planning and Execution in Open-World
- 2025Vision-Language-Policy Model for Dynamic Robot Task Planning