mlsanityData to 7 Oct 2026

ChatVLA: Unified Multimodal Understanding and Robot Control with Vision-Language-Action Model

arXiv 2502.14420 ↗First posted 20 Feb 2025cs.RO cs.CV cs.LGcited by 112

On the map: Vision-language-action models → Reasoning and chain-of-thought in VLA

Abstract (arXiv)

Humans possess a unified cognitive ability to perceive, comprehend, and interact with the physical world. Why can't large language models replicate this holistic understanding? Through a systematic analysis of existing training paradigms in vision-language-action models (VLA), we identify two key challenges: spurious forgetting, where robot training overwrites crucial visual-text alignments, and task interference, where competing control and understanding tasks degrade performance when trained jointly. To overcome these limitations, we propose ChatVLA, a novel framework featuring Phased Alignment Training, which incrementally integrates multimodal data after initial control mastery, and a Mixture-of-Experts architecture to minimize task interference. ChatVLA demonstrates competitive performance on visual question-answering datasets and significantly surpasses state-of-the-art vision-language-action (VLA) methods on multimodal understanding benchmarks. Notably, it achieves a six times higher performance on MMMU and scores 47.2% on MMStar with a more parameter-efficient design than ECoT. Furthermore, ChatVLA demonstrates superior performance on 25 real-world robot manipulation tasks compared to existing VLA methods like OpenVLA. Our findings highlight the potential of our unified framework for achieving both robust multimodal understanding and effective robot control.

Key facts. Cited by 112 papers in the map (data through 7 Oct 2026). Placed in Reasoning and chain-of-thought in VLA, one of 13 directions in Vision-language-action models. Builds on 58 of its 74 references that are in the map. First posted on arXiv 20 Feb 2025.

Builds on

58 of 74 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024CoA-VLA: Improving Vision-Language-Action Models via Visual-Textual Chain-of-Affordance29 Dec
  2. 2024Robotic Control via Embodied Chain-of-Thought Reasoning11 Jul

Other references · 56

  1. 2023Visual Instruction Tuningcited by 7,792
  2. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  5. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  6. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  7. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  8. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  9. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  10. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  11. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  12. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
38 more
  1. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  2. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  3. 2019Towards VQA Models That Can Readcited by 1,696
  4. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  5. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  6. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,449
  7. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  8. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,288
  9. 2023Training Diffusion Models with Reinforcement Learningcited by 939
  10. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 893
  11. 2016A Diagram Is Worth A Dozen Imagescited by 793
  12. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 718
  13. 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 643
  14. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 622
  15. 2023OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Modelscited by 560
  16. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 468
  17. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 466
  18. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
  19. 2024VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Modelscited by 443
  20. 2023An Embodied Generalist Agent in 3D Worldcited by 355
  21. 2023Vision-Language Foundation Models as Effective Robot Imitatorscited by 344
  22. 2024Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Modelscited by 336
  23. 2024TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulationcited by 331
  24. 2023OBELICS: An Open Web-Scale Filtered Dataset of Interleaved Image-Text Documentscited by 316
  25. 2023Zero-Shot Robotic Manipulation with Pretrained Image-Editing Diffusion Modelscited by 264
  26. 2025DexVLA: Vision-Language Model with Plug-In Diffusion Expert for General Robot Controlcited by 191
  27. 2024JanusFlow: Harmonizing Autoregression and Rectified Flow for Unified Multimodal Understanding and Generationcited by 162
  28. 2024Consistency Policy: Accelerated Visuomotor Policies via Consistency Distillationcited by 159
  29. 2024Data Scaling Laws in Imitation Learning for Robotic Manipulationcited by 144
  30. 2024Fine-Tuning of Continuous-Time Diffusion Models as Entropy-Regularized Controlcited by 136
  31. 2024Multimodal Diffusion Transformer: Learning Versatile Behavior from Multimodal Goalscited by 131
  32. 2024ALOHA Unleashed: A Simple Recipe for Robot Dexteritycited by 128
  33. 2023Investigating the Catastrophic Forgetting in Multimodal Large Language Modelscited by 119
  34. 2024MTVQA: Benchmarking Multilingual Text-Centric Visual Question Answeringcited by 93
  35. 2024Mono-InternVL: Pushing the Boundaries of Monolithic Multimodal Large Language Models with Endogenous Visual Pre-trainingcited by 82
  36. 2021AFEC: Active Forgetting of Negative Transfer in Continual Learningcited by 73
  37. 2024The Ingredients for Robotic Diffusion Transformerscited by 72
  38. 2024Diffusion-VLA: Generalizable and Interpretable Robot Foundation Model via Self-Generated Reasoningcited by 57

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 63; 2026: 49.

112 papers.

Sort:

By area:

  1. 2025DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledgecited by 203
  2. 2025Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Bettercited by 142
  3. 2026π0.7: a Steerable Generalist Robotic Foundation Model with Emergent Capabilitiescited by 127
  4. 2025OneTwoVLA: A Unified Vision-Language-Action Model with Adaptive Reasoningcited by 105
  5. 2026VLA-JEPA: Enhancing Vision-Language-Action Model with Latent World Modelcited by 104
  6. 2026StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developingcited by 103
  7. 2025PointVLA: Injecting the 3D World into Vision-Language-Action Modelscited by 101
  8. 2025ForceVLA: Enhancing VLA Models with a Force-aware MoE for Contact-rich Manipulationcited by 87
  9. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 82
  10. 2025Vision-Language-Action (VLA) Models: Concepts, Progress, Applications and Challengescited by 72
  11. 2025InstructVLA: Vision-Language-Action Instruction Tuning from Understanding to Manipulationcited by 71
  12. 2025CogVLA: Cognition-Aligned Vision-Language-Action Model via Instruction-Driven Routing & Sparsificationcited by 70
38 more
  1. 2025RynnVLA-002: A Unified Vision-Language-Action and World Modelcited by 70
  2. 2025Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Surveycited by 66
  3. 2025ChatVLA-2: Vision-Language-Action Model with Open-World Embodied Reasoning from Pretrained Knowledgecited by 49
  4. 2025Pure Vision Language Action (VLA) Models: A Comprehensive Surveycited by 46
  5. 2025Vision Language Action Models in Robotic Manipulation: A Systematic Reviewcited by 38
  6. 2026VLM4VLA: Revisiting Vision-Language-Models in Vision-Language-Action Modelscited by 38
  7. 2025VLA-OS: Structuring and Dissecting Planning Representations and Paradigms in Vision-Language-Action Modelscited by 34
  8. 2025Visual Embodied Brain: Let Multimodal Large Language Models See, Think, and Control in Spacescited by 33
  9. 2025FailSafe: Reasoning and Recovery from Failures in Vision-Language-Action Modelscited by 33
  10. 2025VLSA: Vision-Language-Action Models with Plug-and-Play Safety Constraint Layercited by 32
  11. 2025Exploring the Limits of Vision-Language-Action Manipulations in Cross-task Generalizationcited by 31
  12. 2025Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarkscited by 31
  13. 2025Agentic Robot: A Brain-Inspired Framework for Vision-Language-Action Models in Embodied Agentscited by 30
  14. 2025EO-1: An Open Unified Embodied Foundation Model for General Robot Controlcited by 30
  15. 2025LLaDA-VLA: Vision Language Diffusion Action Modelscited by 30
  16. 2025dVLA: Diffusion Vision-Language-Action Model with Multimodal Chain-of-Thoughtcited by 30
  17. 2025VLA-Arena: An Open-Source Framework for Benchmarking Vision-Language-Action Modelscited by 28
  18. 2025JARVIS-VLA: Post-Training Large-Scale Vision Language Models to Play Visual Games with Keyboards and Mousecited by 27
  19. 2025SIMA 2: A Generalist Embodied Agent for Virtual Worldscited by 26
  20. 2025BEAST: Efficient Tokenization of B-Splines Encoded Action Sequences for Imitation Learningcited by 25
  21. 2025Do What You Say: Steering Vision-Language-Action Models via Runtime Reasoning-Action Alignment Verificationcited by 25
  22. 2025MindDrive: A Vision-Language-Action Model for Autonomous Driving via Online Reinforcement Learningcited by 25
  23. 2025Vlaser: Vision-Language-Action Model with Synergistic Embodied Reasoningcited by 22
  24. 2025Robot-R1: Reinforcement Learning for Enhanced Embodied Reasoning in Roboticscited by 21
  25. 2025ObjectVLA: End-to-End Open-World Object Manipulation Without Demonstrationcited by 20
  26. 2026A Systematic Study of Data Modalities and Strategies for Co-training Large Behavior Models for Robot Manipulationcited by 20
  27. 2025Parallels Between VLA Model Post-Training and Human Motor Learning: Progress, Challenges, and Trendscited by 17
  28. 2025ActiveUMI: Robotic Manipulation with Active Perception from Robot-Free Human Demonstrationscited by 17
  29. 2026LAP: Language-Action Pre-Training Enables Zero-shot Cross-Embodiment Transfercited by 17
  30. 2025Contrastive Representation Regularization for Vision-Language-Action Modelscited by 16
  31. 2026ActiveVLA: Injecting Active Perception into Vision-Language-Action Models for Precise 3D Robotic Manipulationcited by 14
  32. 2026TwinBrainVLA: Unleashing the Potential of Generalist VLMs for Embodied Tasks via Asymmetric Mixture-of-Transformerscited by 14
  33. 2026CycleVLA: Proactive Self-Correcting Vision-Language-Action Models via Subtask Backtracking and Minimum Bayes Risk Decodingcited by 13
  34. 2025Expertise need not monopolize: Action-Specialized Mixture of Experts for Vision-Language-Action Learningcited by 12
  35. 2025NanoVLA: Routing Decoupled Vision-Language Understanding for Nano-sized Generalist Robotic Policiescited by 12
  36. 2025Mantis: A Versatile Vision-Language-Action Model with Disentangled Visual Foresightcited by 12
  37. 2025MM-ACT: Learn from Multimodal Parallel Generation to Actcited by 12
  38. 2026RoboInter: A Holistic Intermediate Representation Suite Towards Robotic Manipulationcited by 12

50 most cited of 112 All 112 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026EmbodiedMidtrain: Bridging the Gap between Vision-Language Models and Vision-Language-Action Models via Mid-training
  2. 2026RoboAlign: Learning Test-Time Reasoning for Language-Action Alignment in Vision-Language-Action Models
  3. 2025Actions as Language: Fine-Tuning VLMs into VLAs Without Catastrophic Forgetting
  4. 2026Does VLA Even Know the Basics? Measuring Commonsense and World Knowledge Retention in Vision-Language-Action Models
  5. 2025Unveiling the Potential of Vision-Language-Action Models with Open-Ended Multimodal Instructions
  6. 2026HMVLA: Hyperbolic Multimodal Fusion for Vision-Language-Action Models
  7. 2024ManipVQA: Injecting Robotic Affordance and Physically Grounded Information into Multi-Modal Large Language Models
  8. 2025PixelVLA: Advancing Pixel-level Understanding in Vision-Language-Action Model