mlsanityData to 7 Oct 2026

RDT2: Exploring the Scaling Limit of UMI Data Towards Zero-Shot Cross-Embodiment Generalization

arXiv 2602.03310 ↗First posted 3 Feb 2026cs.RO cs.AI cs.CVcited by 29

On the map: Vision-language-action models → Generalist VLA models and benchmarks

Abstract (arXiv)

Vision-Language-Action (VLA) models hold promise for generalist robotics but currently struggle with data scarcity, architectural inefficiencies, and the inability to generalize across different hardware platforms. We introduce RDT2, a robotic foundation model built upon a 7B parameter VLM designed to enable zero-shot deployment on novel embodiments for open-vocabulary tasks. To achieve this, we collected one of the largest open-source robotic datasets--over 10,000 hours of demonstrations in diverse families--using an enhanced, embodiment-agnostic Universal Manipulation Interface (UMI). Our approach employs a novel three-stage training recipe that aligns discrete linguistic knowledge with continuous control via Residual Vector Quantization (RVQ), flow-matching, and distillation for real-time inference. Consequently, RDT2 becomes one of the first models that simultaneously zero-shot generalizes to unseen objects, scenes, instructions, and even robotic platforms. Besides, it outperforms state-of-the-art baselines in dexterous, long-horizon, and dynamic downstream tasks like playing table tennis. See https://rdt-robotics.github.io/rdt2/ for more information.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Placed in Generalist VLA models and benchmarks, one of 13 directions in Vision-language-action models. Builds on 68 of its 76 references that are in the map. First posted on arXiv 3 Feb 2026.

Builds on

68 of 76 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025π0.5: a Vision-Language-Action Model with Open-World Generalization22 Apr
  2. 2024π0: A Vision-Language-Action Flow Model for General Robot Control31 Oct
  3. 2024OpenVLA: An Open-Source Vision-Language-Action Model13 Jun
  4. 2024A Survey on Vision-Language-Action Models for Embodied AI23 May
  5. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control28 Jul

Other references · 63

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2020Scaling Laws for Neural Language Modelscited by 6,411
  5. 2022Flow Matching for Generative Modelingcited by 5,426
  6. 2025Qwen2.5-VL Technical Reportcited by 5,418
  7. 2017Neural Discrete Representation Learningcited by 4,650
  8. 2023Qwen Technical Reportcited by 3,273
  9. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  10. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  11. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  12. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
38 more
  1. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  2. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  3. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  4. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
  5. 2021Scaling Vision Transformerscited by 993
  6. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
  7. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 849
  8. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
  9. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 801
  10. 2021SoundStream: An End-to-End Neural Audio Codeccited by 756
  11. 2022Autoregressive Image Generation using Residual Quantizationcited by 706
  12. 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 670
  13. 2021Vector-quantized Image Modeling with Improved VQGANcited by 656
  14. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 634
  15. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 517
  16. 2025RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulationcited by 477
  17. 2021Implicit Behavioral Cloningcited by 442
  18. 2024RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robotscited by 391
  19. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 377
  20. 2024Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robotscited by 373
  21. 2024Latent Action Pretraining from Videoscited by 302
  22. 2022Behavior Transformers: Cloning k modes with one stonecited by 299
  23. 2023RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulationcited by 206
  24. 2022Offline Reinforcement Learning via High-Fidelity Generative Behavior Modelingcited by 177
  25. 2023RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shotcited by 173
  26. 2024RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulationcited by 166
  27. 2024Consistency Policy: Accelerated Visuomotor Policies via Consistency Distillationcited by 159
  28. 2021The Surprising Effectiveness of Representation Learning for Visual Imitationcited by 149
  29. 2024Scaling Proprioceptive-Visual Learning with Heterogeneous Pre-trained Transformerscited by 145
  30. 2024Open-TeleVision: Teleoperation with Immersive Active Visual Feedbackcited by 137
  31. 2020Language-Conditioned Imitation Learning for Robot Manipulation Taskscited by 136
  32. 2025RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concretecited by 134
  33. 2023RoboVQA: Multimodal Long-Horizon Reasoning for Roboticscited by 116
  34. 2025EgoVLA: Learning Vision-Language-Action Models from Egocentric Human Videoscited by 116
  35. 2025HD-EPIC: A Highly-Detailed Egocentric Video Datasetcited by 101
  36. 2021gradSim: Differentiable simulation for system identification and visuomotor controlcited by 100
  37. 2025Being-H0: Vision-Language-Action Pretraining from Large-Scale Human Videoscited by 94
  38. 2024One-Step Diffusion Policy: Fast Visuomotor Policies via Diffusion Distillationcited by 83

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 29.

29 papers.

Sort:

By area:

  1. 2026ABot-M0: VLA Foundation Model for Robotic Manipulation with Action Manifold Learningcited by 94
  2. 2026World Action Models: The Next Frontier in Embodied AIcited by 46
  3. 2026Xiaomi-Robotics-0: An Open-Sourced Vision-Language-Action Model with Real-Time Executioncited by 42
  4. 2026Xiaomi-Robotics-1: Scaling Vision-Language-Action Models with over 100K Hours of Real-World Trajectoriescited by 24
  5. 2026FASTER: Rethinking Real-Time Flow VLAscited by 20
  6. 2026FRAPPE: Infusing World Modeling into Generalist Policies via Multiple Future Representation Alignmentcited by 19
  7. 2026EventVLA: Event-Driven Visual Evidence Memory for Long-Horizon Vision-Language-Action Policiescited by 18
  8. 2026LAP: Language-Action Pre-Training Enables Zero-shot Cross-Embodiment Transfercited by 17
  9. 2026Realtime-VLA FLASH: Speculative Inference Framework for Diffusion-based VLAscited by 8
  10. 2026ACE-Ego-0: Unifying Egocentric Human and Robotic Data for VLA Pretrainingcited by 8
  11. 2026LaST-R1: Reinforcing Robotic Manipulation via Adaptive Physical Latent Reasoningcited by 7
  12. 2026IntentVLA: Short-Horizon Intent Modeling for Aliased Robot Manipulationcited by 7
17 more
  1. 2026GEM: Generative Supervision Helps Embodied Intelligencecited by 6
  2. 2026X-Tokenizer: A Multimodal Action Tokenizer for Vision-Language-Action Pretrainingcited by 6
  3. 2026Motus2: A Self-Evolving General World Model for Dexterous Manipulationcited by 6
  4. 2026UMI-3D: Extending Universal Manipulation Interface from Vision-Limited to 3D Spatial Perceptioncited by 4
  5. 2026Data Pyramid for Embodied Manipulation: A Surveycited by 4
  6. 2026Cloak: Zero-Shot Cross-Embodiment Manipulation by Masking the End-Effector from the VLAcited by 3
  7. 2026Knowing When to Stop: Adaptive Action Chunking via Internal Cross-Attention Dynamics in VLAscited by 3
  8. 2026VISTA: Vision-Grounded and Physics-Validated Adaptation of UMI data for VLA Trainingcited by 2
  9. 2026Towards Predictive, Aligned, and Scalable Robot Learningcited by 2
  10. 2026HiFi-UMI: Learning Deployable Manipulation Policies from High-Fidelity UMI Data Alonecited by 2
  11. 2026Cortex 2.0: Grounding World Models in Real-World Industrial Deploymentcited by 1
  12. 2026DyPES-VLA: Learning Shared Dynamics Priors and Embodiment-Specific Control for Cross-Embodiment Manipulationcited by 1
  13. 2026HybridFlow: A 2-NFE Generative Policy for Real-Time Robotic Manipulationcited by 0
  14. 2026PACT: Self-Evolving Physical Safety Alignment for Diffusion Policies in Embodied Manipulationcited by 0
  15. 2026ZETA: A Controlled Study of Zero-Shot Cross-Embodiment VLA Transfer for Tabletop Manipulationcited by 0
  16. 2026EgoHumanoid-V2: Human-to-Humanoid Transfer of Coordinated Whole-Body Skills for Loco-Manipulationcited by 0
  17. 2026RoboAware: Learning to Coordinate Embodied Skills from Counterfactual Outcomescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Exploring the Limits of Vision-Language-Action Manipulations in Cross-task Generalization
  2. 2025GigaBrain-0: A World Model-Powered Vision-Language-Action Model
  3. 2026Colosseum V2: Benchmarking Generalization for Vision-Language-Action Models
  4. 2025REALM: A Real-to-Sim Validated Benchmark for Generalization in Robotic Manipulation
  5. 2025From Seeing to Doing: Bridging Reasoning and Decision for Robotic Manipulation
  6. 2026Vision-Language-Action in Robotics: A Survey of Datasets, Benchmarks, and Data Engines
  7. 2026JoyAI-RA 0.1: A Foundation Model for Robotic Autonomy
  8. 2025Goal-VLA: Image-Generative VLMs as Object-Centric World Models Empowering Zero-shot Robot Manipulation