mlsanityData to 7 Oct 2026

RVT-2: Learning Precise Manipulation from Few Demonstrations

arXiv 2406.08545 ↗First posted 12 Jun 2024cs.RO cs.AI cs.CVcited by 135

Not on the map. Why?

Cited by 47 in Vision-language-action models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In this work, we study how to build a robotic system that can solve multiple 3D manipulation tasks given language instructions. To be useful in industrial and household domains, such a system should be capable of learning new tasks with few demonstrations and solving them precisely. Prior works, like PerAct and RVT, have studied this problem, however, they often struggle with tasks requiring high precision. We study how to make them more effective, precise, and fast. Using a combination of architectural and system-level improvements, we propose RVT-2, a multitask 3D manipulation model that is 6X faster in training and 2X faster in inference than its predecessor RVT. RVT-2 achieves a new state-of-the-art on RLBench, improving the success rate from 65% to 82%. RVT-2 is also effective in the real world, where it can learn tasks requiring high precision, like picking up and inserting plugs, with just 10 demonstrations. Visual results, code, and trained model are provided at: https://robotic-view-transformer-2.github.io/.

Key facts. Cited by 135 papers in the map (data through 7 Oct 2026). Builds on 48 of its 53 references that are in the map. First posted on arXiv 12 Jun 2024.

Builds on

48 of 53 references.

References · 48

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  3. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  4. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  5. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  6. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  7. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  8. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  9. 2016Multi-View 3D Object Detection Network for Autonomous Drivingcited by 721
  10. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
  11. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 598
  12. 2015Multi-view Convolutional Neural Networks for 3D Shape Recognitioncited by 564
36 more
  1. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 545
  2. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 517
  3. 20218-bit Optimizers via Block-wise Quantizationcited by 395
  4. 2020Transporter Networks: Rearranging the Visual World for Robotic Manipulationcited by 342
  5. 2023RVT: Robotic View Transformer for 3D Object Manipulationcited by 224
  6. 2023MimicPlay: Long-Horizon Imitation Learning by Watching Human Playcited by 214
  7. 2018Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Taskscited by 150
  8. 2021Revisiting Point Cloud Shape Classification with a Simple and Effective Baselinecited by 145
  9. 2020MVTN: Multi-View Transformation Network for 3D Shape Recognitioncited by 140
  10. 2022Multi-View Transformer for 3D Visual Groundingcited by 138
  11. 2023Act3D: 3D Feature Field Transformers for Multi-Task Robotic Manipulationcited by 135
  12. 2021Coarse-to-Fine Q-attention: Efficient Learning for Visual Robotic Manipulation via Discretisationcited by 120
  13. 2022Instruction-driven history-aware policies for robotic manipulationscited by 93
  14. 2020Transformers for One-Shot Visual Imitationcited by 77
  15. 2021Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformerscited by 71
  16. 2023Waypoint-Based Imitation Learning for Robotic Manipulationcited by 59
  17. 2019Deep Reinforcement Learning for Industrial Insertion Tasks with Visual Inputs and Natural Rewardscited by 58
  18. 2021StructFormer: Learning Spatial Structure for Language-Guided Semantic Rearrangement of Novel Objectscited by 57
  19. 2023PolarNet: 3D Point Clouds for Language-Guided Robotic Manipulationcited by 56
  20. 2022Look Closer: Bridging Egocentric and Third-Person Views with Transformers for Robotic Manipulationcited by 49
  21. 2021Tactile-RL for Insertion: Generalization to Objects of Unknown Geometrycited by 45
  22. 2023IndustReal: Transferring Contact-Rich Assembly Tasks from Simulation to Realitycited by 44
  23. 2022IFOR: Iterative Flow Minimization for Robotic Object Rearrangementcited by 42
  24. 2023M2T2: Multi-Task Masked Transformer for Object-centric Pick and Placecited by 40
  25. 2023KITE: Keypoint-Conditioned Policies for Semantic Manipulationcited by 39
  26. 2024Fourier Transporter: Bi-Equivariant Robotic Manipulation in 3Dcited by 32
  27. 2021Transformer-based deep imitation learning for dual-arm robot manipulationcited by 31
  28. 2023Shelving, Stacking, Hanging: Relational Pose Diffusion for Multi-modal Rearrangementcited by 29
  29. 2021Differentiable Spatial Planning using Transformerscited by 25
  30. 2021Voint Cloud: Multi-View Point Cloud Representation for 3D Understandingcited by 22
  31. 2021Rendering Point Clouds with Compute Shaders and Vertex Order Optimizationcited by 14
  32. 2021Assistive Tele-op: Leveraging Transformers to Collect Robotic Task Demonstrationscited by 11
  33. 2020PackIt: A Virtual Environment for Geometric Planningcited by 10
  34. 2021Motion Planning Transformers: A Motion Planning Framework for Mobile Robotscited by 10
  35. 2020Understanding Multi-Modal Perception Using Behavioral Cloning for Peg-In-a-Hole Insertion Taskscited by 8
  36. 2020Imitation Learning for High Precision Peg-in-Hole Taskscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 15; 2025: 67; 2026: 53.

135 papers.

Sort:

By area:

  1. 2024ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulationcited by 305
  2. 2025HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Modelcited by 176
  3. 2025HAMSTER: Hierarchical Action Models For Open-World Robot Manipulationcited by 125
  4. 2025GraspVLA: a Grasping Foundation Model Pre-trained on Billion-scale Synthetic Action Datacited by 120
  5. 2025PointVLA: Injecting the 3D World into Vision-Language-Action Modelscited by 101
  6. 2024Manipulate-Anything: Automating Real-World Robots using Vision-Language Modelscited by 82
  7. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 82
  8. 2025BridgeVLA: Input-Output Alignment for Efficient 3D Manipulation Learning with Vision-Language Modelscited by 77
  9. 2024Towards Synergistic, Generalized, and Efficient Dual-System for Robotic Manipulationcited by 71
  10. 2025GeoVLA: Empowering 3D Representations in Vision-Language-Action Modelscited by 68
  11. 2024Code-as-Monitor: Constraint-aware Visual Programming for Reactive and Proactive Robotic Failure Detectioncited by 59
  12. 2025EnerVerse: Envisioning Embodied Future Space for Robotics Manipulationcited by 59
38 more
  1. 2025Evo-0: Vision-Language-Action Model with Implicit Spatial Understandingcited by 55
  2. 2025VLA-0: Building State-of-the-Art VLAs with Zero Modificationcited by 49
  3. 2025Pure Vision Language Action (VLA) Models: A Comprehensive Surveycited by 46
  4. 2024Lift3D Foundation Policy: Lifting 2D Large-Scale Pretrained Models for Robust 3D Robotic Manipulationcited by 45
  5. 2024Dream to Manipulate: Compositional World Models Empowering Robot Imitation Learning with Imaginationcited by 45
  6. 2025ManiFlow: A General Robot Manipulation Policy via Consistency Flow Trainingcited by 43
  7. 2024Towards Generalizable Vision-Language Robotic Manipulation: A Benchmark and LLM-guided 3D Policycited by 37
  8. 2025GWM: Towards Scalable Gaussian World Models for Robotic Manipulationcited by 37
  9. 2024Deep Generative Models in Robotics: A Survey on Learning from Multimodal Demonstrationscited by 35
  10. 2024G3Flow: Generative 3D Semantic Flow for Pose-aware and Generalizable Object Manipulationcited by 35
  11. 2025Exploring the Limits of Vision-Language-Action Manipulations in Cross-task Generalizationcited by 31
  12. 2025LLaDA-VLA: Vision Language Diffusion Action Modelscited by 30
  13. 2024SPA: 3D Spatial-Awareness Enables Effective Embodied Representationcited by 28
  14. 2025Pre-training Auto-regressive Robotic Models with 4D Representationscited by 27
  15. 2024Autoregressive Action Sequence Learning for Robotic Manipulationcited by 26
  16. 2024In-Context Learning Enables Robot Action Prediction in LLMscited by 25
  17. 2024TacDiffusion: Force-domain Diffusion Policy for Precise Tactile Manipulationcited by 23
  18. 2025UAD: Unsupervised Affordance Distillation for Generalization in Robotic Manipulationcited by 23
  19. 2024VoxAct-B: Voxel-Based Acting and Stabilizing Policy for Bimanual Manipulationcited by 21
  20. 2025Enhancing Generalization in Vision-Language-Action Models by Preserving Pretrained Representationscited by 21
  21. 2026HoloBrain-0 Technical Reportcited by 20
  22. 2025QDepth-VLA: Quantized Depth Prediction as Auxiliary Supervision for Vision-Language-Action Modelscited by 19
  23. 2025Parallels Between VLA Model Post-Training and Human Motor Learning: Progress, Challenges, and Trendscited by 17
  24. 2025Seeing to Act, Prompting to Specify: A Bayesian Factorization of Vision Language Action Policycited by 17
  25. 2025Adapt3R: Adaptive 3D Scene Representation for Domain Transfer in Imitation Learningcited by 16
  26. 2025Chain-of-Action: Trajectory Autoregressive Modeling for Robotic Manipulationcited by 16
  27. 2025GLOVER++: Unleashing the Potential of Affordance Learning from Human Behaviors for Robotic Manipulationcited by 14
  28. 2025Do You Know Where Your Camera Is? View-Invariant Policy Learning with Camera Conditioningcited by 14
  29. 2026ActiveVLA: Injecting Active Perception into Vision-Language-Action Models for Precise 3D Robotic Manipulationcited by 14
  30. 2025OG-VLA: Orthographic Image Generation for 3D-Aware Vision-Language Action Modelcited by 13
  31. 2025FlowRAM: Grounding Flow Matching Policy with Region-Aware Mamba Framework for Robotic Manipulationcited by 13
  32. 2025GP3: A 3D Geometry-Aware Policy with Multi-View Images for Robotic Manipulationcited by 11
  33. 2024Effective Tuning Strategies for Generalist Robot Manipulation Policiescited by 10
  34. 2025AnyPlace: Learning Generalized Object Placement for Robot Manipulationcited by 10
  35. 2025BFA: Best-Feature-Aware Fusion for Multi-View Fine-grained Manipulationcited by 10
  36. 2025Neural Brain: A Neuroscience-inspired Framework for Embodied Agentscited by 10
  37. 2025GAF: Gaussian Action Field as a 4D Representation for Dynamic World Modeling in Robotic Manipulationcited by 10
  38. 2025ManiGaussian++: General Robotic Bimanual Manipulation with Hierarchical Gaussian World Modelcited by 10

50 most cited of 135 All 135 (JSON)