Decoupling Vision, Language, and Action for Efficient Multi-Task Robot Policies
On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models
also in 1 more
Abstract (arXiv)
Vision-Language-Action (VLA) policies commonly run Vision-Language Model (VLM) backbones with billions of parameters at every policy inference, which costs latency and energy. We revisit a decoupled alternative for multi-task manipulation: separate vision and language encoders whose representations condition a compact action head. We run a standardized comparison that varies the vision encoder, the language encoder, and the action head while holding the demonstrations, the training-step budget, the tasks, the evaluation protocol, and the measurement platform fixed, against seven VLA baselines. The resulting Decoupled Embodiment Model (DEM) combines a fine-tuned DINOv3 vision encoder, a frozen NeoBERT language encoder, and a MeanFlow head that generates an action chunk in one forward pass. On 18 RoboCasa tasks evaluated with held-out instruction paraphrases and randomized scenes, DEM reaches 55.6% mean success against 56.9% for GR00T N1.7 and 54.6% for π0.5, and on three real-robot tasks it reaches 66.0% against 68.0% for GR00T N1.7. On the same workstation, DEM needs 6.1,ms per policy forward pass, a maximum throughput of 162.7 policy calls per second, and draws an estimated 2.07,J of GPU energy per call, eight to seventeen times the throughput and six to fifteen times less energy than these VLM-backbone policies. Within this trained-task regime, DEM sits on the observed success--latency--energy frontier and provides a strong, efficient baseline for language-conditioned robot skills.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 44 of its 48 references that are in the map. First posted on arXiv 16 Sep 2026.
Builds on
44 of 48 references.
Closest ancestors · 5
Same direction, newest first
- 2026DAM-VLA: Decoupled Asynchronous Multimodal Vision Language Action model10 Jun
- 2025SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics2 Jun
- 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success27 Feb
- 2025Improving Vision-Language-Action Model with Online Reinforcement Learning28 Jan
- 2024DeeR-VLA: Dynamic Inference of Multimodal Large Language Models for Efficient Robot Execution4 Nov
Other references · 39
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
- 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
- 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
- 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
- 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,761
- 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
- 2024Octo: An Open-Source Generalist Robot Policycited by 1,484
- 2025DINOv3cited by 1,414
- 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,225
27 more
- 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
- 2025Mean Flows for One-step Generative Modelingcited by 561
- 2024Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inferencecited by 554
- 2024RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robotscited by 407
- 2024TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulationcited by 339
- 2025π*0.6: a VLA That Learns From Experiencecited by 303
- 2023Where are we in the search for an Artificial Visual Cortex for Embodied Intelligence?cited by 218
- 2025Real-Time Execution of Action Chunking Flow Policiescited by 216
- 2025Steering Your Diffusion Policy with Latent Space Reinforcement Learningcited by 169
- 2023FurnitureBench: Reproducible Real-World Benchmark for Long-Horizon Complex Manipulationcited by 133
- 2025VLA-Cache: Efficient Vision-Language-Action Manipulation via Adaptive Token Cachingcited by 86
- 2024BAKU: An Efficient Transformer for Multi-Task Policy Learningcited by 80
- 2025mmBERT: A Modern Multilingual Encoder with Annealed Language Learningcited by 45
- 2025Seq vs Seq: An Open Suite of Paired Encoders and Decoderscited by 38
- 2025MP1: MeanFlow Tames Policy Learning in 1-step for Robotic Manipulationcited by 35
- 2025RLinf-VLA: A Unified and Efficient Framework for Reinforcement Learning of Vision-Language-Action Modelscited by 28
- 2025NeoBERT: A Next-Generation BERTcited by 16
- 2026TurboVLA: Real-Time Vision-Language-Action Model at 32 Hz on an RTX 4090 with <1 GB VRAMcited by 10
- 2025Dynamic Rank Adjustment in Diffusion Policies for Efficient and Flexible Trainingcited by 7
- 2026Mean-Flow based One-Step Vision-Language-Actioncited by 7
- 2025PRISM-DP: Spatial Pose-based Observations for Diffusion-Policies via Segmentation, Mesh Generation, and Pose Trackingcited by 5
- 2026Artificial Foveated Perception for Mitigating Shortcut Learning in Robotic Foundation Modelscited by 5
- 2024Optimizing Active Perception for Learning Simultaneous Viewpoint Selection and Manipulation with Diffusion Policycited by 3
- 2025Hybrid Diffusion Policies with Projective Geometric Algebra for Efficient Robot Manipulation Learningcited by 3
- 2026ReactVLA: Fast and Lightweight Reactive Robot Manipulation via Improved Mean Flow Action Generationcited by 3
- 2026MINERVA: How Small Can a Manipulation Policy Be and Still Solve LIBERO?cited by 3
- 2025Subsecond 3D Mesh Generation for Robot Manipulationcited by 2
Built on by
Citing papers per year: 2026: 2.
2 papers.
- 2026vla.simd: Efficient CPU Inference for Language-Conditioned Manipulationcited by 0
- 2026Dexterous Tactile World Modelcited by 0
- 2026Dexterous Tactile World Model28 Sep 2026
- 2026vla.simd: Efficient CPU Inference for Language-Conditioned Manipulation21 Sep 2026
Similar, not cited
Close in wording, no citation either way
- 2026Reducing Temporal Redundancy for Efficient Vision-Language-Action Inference
- 2026Rethinking Visual-Language-Action Model Scaling: Alignment, Mixture, and Regularization
- 2025VLA-Adapter: An Effective Paradigm for Tiny-Scale Vision-Language-Action Model
- 2026G0.5: One Autoregressive Stream for Robot Reasoning and Action
- 2026Vision-Language-Action in Robotics: A Survey of Datasets, Benchmarks, and Data Engines
- 2025An Anatomy of Vision-Language-Action Models: From Modules to Milestones and Challenges
- 2026CoRE-VLA: Towards Scalable and Robust Vision-Language-Action Modeling via Conditional Routing of Experts
- 2025Efficient Vision-Language-Action Models for Embodied Manipulation: A Systematic Survey