mlsanityData to 7 Oct 2026

Towards Synergistic, Generalized, and Efficient Dual-System for Robotic Manipulation

arXiv 2410.08001 ↗First posted 10 Oct 2024cs.RO cs.AIcited by 71

On the map: Vision-language-action models → Generalist VLA models and benchmarks

Abstract (arXiv)

The increasing demand for versatile robotic systems to operate in diverse and dynamic environments has emphasized the importance of a generalist policy, which leverages a large cross-embodiment data corpus to facilitate broad adaptability and high-level reasoning. However, the generalist would struggle with inefficient inference and cost-expensive training. The specialist policy, instead, is curated for specific domain data and excels at task-level precision with efficiency. Yet, it lacks the generalization capacity for a wide range of applications. Inspired by these observations, we introduce RoboDual, a synergistic dual-system that supplements the merits of both generalist and specialist policy. A diffusion transformer-based specialist is devised for multi-step action rollouts, exquisitely conditioned on the high-level task understanding and discretized action output of a vision-language-action (VLA) based generalist. Compared to OpenVLA, RoboDual achieves 26.7% improvement in real-world setting and 12% gain on CALVIN by introducing a specialist policy with merely 20M trainable parameters. It maintains strong performance with 5% of demonstration data only, and enables a 3.8 times higher control frequency in real-world deployment. Code would be made publicly available. Our project page is hosted at: https://opendrivelab.com/RoboDual/

Key facts. Cited by 71 papers in the map (data through 7 Oct 2026). Placed in Generalist VLA models and benchmarks, one of 13 directions in Vision-language-action models. Builds on 70 of its 74 references that are in the map. First posted on arXiv 10 Oct 2024.

Builds on

70 of 74 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024VLATest: Testing and Evaluating Vision-Language-Action Models for Robotic Manipulation19 Sep
  2. 2024OpenVLA: An Open-Source Vision-Language-Action Model13 Jun
  3. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control28 Jul

Other references · 67

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2020Denoising Diffusion Implicit Modelscited by 9,655
  7. 2023Visual Instruction Tuningcited by 7,792
  8. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  9. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  10. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  11. 2022Scalable Diffusion Models with Transformerscited by 5,931
  12. 2022Classifier-Free Diffusion Guidancecited by 5,717
38 more
  1. 2017Neural Discrete Representation Learningcited by 4,650
  2. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  3. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  4. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  5. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  6. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  7. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  8. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  9. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  10. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  11. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  12. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  13. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  14. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
  15. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 849
  16. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 801
  17. 2023LISA: Reasoning Segmentation via Large Language Modelcited by 770
  18. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 760
  19. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 634
  20. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 598
  21. 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 564
  22. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 545
  23. 2023Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulationcited by 353
  24. 2023Vision-Language Foundation Models as Effective Robot Imitatorscited by 344
  25. 2024Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Modelscited by 336
  26. 20243D-VLA: A 3D Vision-Language-Action Generative World Modelcited by 320
  27. 20243D Diffuser Actor: Policy Diffusion with 3D Scene Representationscited by 278
  28. 2023Zero-Shot Robotic Manipulation with Pretrained Image-Editing Diffusion Modelscited by 264
  29. 2021A Survey of Embodied AI: From Simulators to Research Taskscited by 256
  30. 2023Large Language Models as Commonsense Knowledge for Large-Scale Task Planningcited by 231
  31. 2023RVT: Robotic View Transformer for 3D Object Manipulationcited by 224
  32. 2020Language Conditioned Imitation Learning over Unstructured Datacited by 208
  33. 2023OneLLM: One Framework to Align All Modalities with Languagecited by 185
  34. 2024Consistency Policy: Accelerated Visuomotor Policies via Consistency Distillationcited by 159
  35. 2022What Matters in Language Conditioned Robotic Imitation Learning over Unstructured Datacited by 137
  36. 2023Act3D: 3D Feature Field Transformers for Multi-Task Robotic Manipulationcited by 135
  37. 2024RVT-2: Learning Precise Manipulation from Few Demonstrationscited by 135
  38. 2024Multimodal Diffusion Transformer: Learning Versatile Behavior from Multimodal Goalscited by 131

50 most cited of 67 All 67 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 34; 2026: 36.

71 papers.

Sort:

By area:

  1. 2025AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systemscited by 390
  2. 2025UniVLA: Learning to Act Anywhere with Task-centric Latent Actionscited by 386
  3. 2025DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledgecited by 203
  4. 2025VLA-Adapter: An Effective Paradigm for Tiny-Scale Vision-Language-Action Modelcited by 140
  5. 2025A Survey on Vision-Language-Action Models: An Action Tokenization Perspectivecited by 96
  6. 2025ForceVLA: Enhancing VLA Models with a Force-aware MoE for Contact-rich Manipulationcited by 87
  7. 2025OpenHelix: A Short Survey, Empirical Analysis, and Open-Source Dual-System VLA Model for Robotic Manipulationcited by 79
  8. 2025InstructVLA: Vision-Language-Action Instruction Tuning from Understanding to Manipulationcited by 71
  9. 2025Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Surveycited by 66
  10. 2025WorldEval: World Model as Real-World Robot Policies Evaluatorcited by 64
  11. 2024Diffusion-VLA: Generalizable and Interpretable Robot Foundation Model via Self-Generated Reasoningcited by 57
  12. 2025Hume: Introducing System-2 Thinking in Visual-Language-Action Modelcited by 56
38 more
  1. 2025FLOWER: Democratizing Generalist Robot Policies with Efficient Vision-Language-Action Flow Policiescited by 56
  2. 2025ChatVLA-2: Vision-Language-Action Model with Open-World Embodied Reasoning from Pretrained Knowledgecited by 49
  3. 2026InternVLA-A1: Unifying Understanding, Generation and Action for Robotic Manipulationcited by 48
  4. 2026AHA-WAM:Asynchronous Horizon-Adaptive World-Action Modeling with Observation-Guided Context Routingcited by 41
  5. 2025EnerVerse-AC: Envisioning Embodied Environments with Action Conditioncited by 40
  6. 2025A Survey on Efficient Vision-Language-Action Modelscited by 34
  7. 2025Efficient Vision-Language-Action Models for Embodied Manipulation: A Systematic Surveycited by 25
  8. 2026Global Prior Meets Local Consistency: Dual-Memory Augmented Vision-Language-Action Model for Efficient Robotic Manipulationcited by 19
  9. 2025Seeing to Act, Prompting to Specify: A Bayesian Factorization of Vision Language Action Policycited by 17
  10. 2025RoVer: Robot Reward Model as Test-Time Verifier for Vision-Language-Action Modelcited by 16
  11. 2025MergeVLA: Cross-Skill Model Merging Toward a Generalist Vision-Language-Action Agentcited by 15
  12. 2025Video2Act: A Dual-System Video Diffusion Policy with Robotic Spatio-Motional Modelingcited by 13
  13. 2025RationalVLA: A Rational Vision-Language-Action Model with Dual Systemcited by 12
  14. 2025Unifying Perception and Action: A Hybrid-Modality Pipeline with Implicit Visual Chain-of-Thought for Robotic Action Generationcited by 12
  15. 2025PointMapPolicy: Structured Point Cloud Processing for Multi-Modal Imitation Learningcited by 10
  16. 2026MMaDA-VLA: Large Diffusion Vision-Language-Action Model with Unified Multi-Modal Instruction and Generationcited by 10
  17. 2025ETA: Efficiency through Thinking Ahead, A Dual Approach to Self-Driving with Large Modelscited by 9
  18. 2026MemoryVLA++: Temporal Modeling via Memory and Imagination in Vision-Language-Action Modelscited by 9
  19. 2025Compose Your Policies! Improving Diffusion-based or Flow-based Robot Policies via Test-time Distribution-level Compositioncited by 8
  20. 2025Survey of Vision-Language-Action Models for Embodied Manipulationcited by 7
  21. 2026AsyncVLA: An Asynchronous VLA for Fast and Robust Navigation on the Edgecited by 7
  22. 2026HiVLA: A Visual-Grounded-Centric Hierarchical Embodied Manipulation Systemcited by 7
  23. 2025HyperVLA: Efficient Inference in Vision-Language-Action Models via Hypernetworkscited by 6
  24. 2025Asynchronous Fast-Slow Vision-Language-Action Policies for Whole-Body Robotic Manipulationcited by 6
  25. 2025Learning to See and Act: Task-Aware Virtual View Exploration for Robotic Manipulationcited by 5
  26. 2025Toward Embodiment Equivariant Vision-Language-Action Policycited by 5
  27. 2025PhysiAgent: An Embodied Agent Framework in Physical Worldcited by 5
  28. 2026UAOR: Uncertainty-aware Observation Reinjection for Vision-Language-Action Modelscited by 5
  29. 2026From Abstraction to Instantiation: Learning Behavioral Representation for Vision-Language-Action Modelcited by 5
  30. 2026APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policiescited by 5
  31. 2026TapSampling: Inference-Time Sampling with a Task-Progress-Understanding Verifier for Robotic Manipulationcited by 4
  32. 2026Cortex: A Bidirectionally Aligned Embodied Agent Framework for Long-horizon Manipulationcited by 4
  33. 2026Generalizable VLA Finetuning via Representation Anchoring and Language-Action Alignmentcited by 4
  34. 2026LABSHIELD: A Multimodal Benchmark for Safety-Critical Reasoning and Planning in Scientific Laboratoriescited by 3
  35. 2026Look Before Acting: Enhancing Vision Foundation Representations for Vision-Language-Action Modelscited by 3
  36. 2026OASIS: Observation-Action Space Alignment via SE(3) Trajectory Prediction for Robotic Manipulationcited by 3
  37. 2025IntentionVLA: Generalizable and Efficient Embodied Intention Reasoning for Human-Robot Interactioncited by 2
  38. 2026VISTA: Enhancing Visual Conditioning via Track-Following Preference Optimization in Vision-Language-Action Modelscited by 2

50 most cited of 71 All 71 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026ManipArena: Comprehensive Real-world Evaluation of Reasoning-Oriented Generalist Robot Manipulation
  2. 2026RDT2: Exploring the Scaling Limit of UMI Data Towards Zero-Shot Cross-Embodiment Generalization
  3. 2023RoboCat: A Self-Improving Generalist Agent for Robotic Manipulation
  4. 2025A Taxonomy for Evaluating Generalist Robot Manipulation Policies
  5. 2024RH20T-P: A Primitive-Level Robotic Dataset Towards Composable Generalization Agents
  6. 2025RoBridge: A Hierarchical Architecture Bridging Cognition and Execution for General Robotic Manipulation
  7. 2026Grounding Sim-to-Real Generalization in Robotic Manipulation: An Empirical Study with Vision-Language-Action Models
  8. 2025REALM: A Real-to-Sim Validated Benchmark for Generalization in Robotic Manipulation