VLASCD: A Visual Language Action Model for Simultaneous Chatting and Decision Making
Not on the map. Why?
Abstract (arXiv)
Recent large pretrained models such as LLMs (e.g., GPT series) and VLAs (e.g., OpenVLA) have achieved notable progress on multimodal tasks, yet they are built upon a multi-input single-output (MISO) paradigm. We show that this paradigm fundamentally limits performance in multi-input multi-output (MIMO) scenarios, where parallel task execution is required. In MISO architectures, tasks compete for a shared output channel, creating mutual exclusion effects that cause unbalanced optimization and degraded performance. To address this gap, we introduce MIMO-VLA (VLASCD), a unified training framework that enables concurrent multi-task outputs, exemplified by simultaneous dialogue generation and decision-making. Inspired by human cognition, MIMO-VLA eliminates interference between tasks and supports efficient parallel processing. Experiments on the CARLA autonomous driving platform demonstrate that MIMO-VLA substantially outperforms state-of-the-art MISO-based LLMs, reinforcement learning models, and VLAs in MIMO settings, establishing a new direction for multimodal and multitask learning.
Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 29 of its 46 references that are in the map. First posted on arXiv 21 Oct 2024.
Builds on
29 of 46 references.
References · 29
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
- 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
- 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
- 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
- 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
- 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
17 more
- 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
- 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 565
- 2023Is ChatGPT a Good NLG Evaluator? A Preliminary Studycited by 422
- 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 407
- 2022Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)cited by 402
- 2023An Embodied Generalist Agent in 3D Worldcited by 365
- 20243D-VLA: A 3D Vision-Language-Action Generative World Modelcited by 328
- 2023Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Drivingcited by 232
- 2023Is ChatGPT a Good Recommender? A Preliminary Studycited by 193
- 2023LanguageMPC: Large Language Models as Decision Makers for Autonomous Drivingcited by 190
- 2019Multimodal End-to-End Autonomous Drivingcited by 110
- 2019Model-free Deep Reinforcement Learning for Urban Autonomous Drivingcited by 98
- 2018Optimizing Agent Behavior over Long Time Scales by Transporting Valuecited by 58
- 2022Enhance Sample Efficiency and Robustness of End-to-end Urban Autonomous Driving via Semantic Masked World Modelcited by 45
- 2022Flow-based Recurrent Belief State Learning for POMDPscited by 16
- 2024Efficient and Generalized end-to-end Autonomous Driving System with Latent Deep Reinforcement Learning and Demonstrationscited by 7
- 2024A Framework to Implement 1+N Multi-task Fine-tuning Pattern in LLMs Using the CGC-LORA Algorithmcited by 1
Built on by
Citing papers per year: 2025: 3; 2026: 2.
5 papers.
- 2025JanusVLN: Decoupling Semantics and Spatiality with Dual Implicit Memory for Vision-Language Navigationcited by 89
- 2025SwiftVLA: Unlocking Spatiotemporal Dynamics for Lightweight VLA Models at Minimal Overheadcited by 26
- 2026One Token Per Frame: Reconsidering Visual Bandwidth in World Models for VLA Policycited by 6
- 2025End-to-end Listen, Look, Speak and Actcited by 5
- 2026ALAM: Algebraically Consistent Latent Action Model for Vision-Language-Action Modelscited by 3
- 2026ALAM: Algebraically Consistent Latent Action Model for Vision-Language-Action Models11 May 2026
- 2026One Token Per Frame: Reconsidering Visual Bandwidth in World Models for VLA Policy8 May 2026
- 2025SwiftVLA: Unlocking Spatiotemporal Dynamics for Lightweight VLA Models at Minimal Overhead30 Nov 2025
- 2025End-to-end Listen, Look, Speak and Act19 Oct 2025
- 2025JanusVLN: Decoupling Semantics and Spatiality with Dual Implicit Memory for Vision-Language Navigation26 Sep 2025
Similar, not cited
Close in wording, no citation either way
- 2025ChatVLA: Unified Multimodal Understanding and Robot Control with Vision-Language-Action Model
- 2025VLAs are Confined yet Capable of Generalizing to Novel Instructions
- 2023Accountable Textual-Visual Chat Learns to Reject Human Instructions in Image Re-creation
- 2026The Ebb and Flow of Multimodal Focus: Scheduling Visual Relay Windows for Grounded VLM Reasoning
- 2025MM-ACT: Learn from Multimodal Parallel Generation to Act
- 2025RLinf-VLA: A Unified and Efficient Framework for Reinforcement Learning of Vision-Language-Action Models
- 2026UAM: A Dual-Stream Perspective on Forgetting in VLA Training
- 2024A Survey on Vision-Language-Action Models for Embodied AI