mlsanityData to 7 Oct 2026

Adapt3R: Adaptive 3D Scene Representation for Domain Transfer in Imitation Learning

arXiv 2503.04877 ↗First posted 6 Mar 2025cs.CV cs.AI cs.LGcited by 16

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

Imitation Learning can train robots to perform complex and diverse manipulation tasks, but learned policies are brittle with observations outside of the training distribution. 3D scene representations that incorporate observations from calibrated RGBD cameras have been proposed as a way to mitigate this, but in our evaluations with unseen embodiments and camera viewpoints they show only modest improvement. To address those challenges, we propose Adapt3R, a general-purpose 3D observation encoder which synthesizes data from calibrated RGBD cameras into a vector that can be used as conditioning for arbitrary IL algorithms. The key idea is to use a pretrained 2D backbone to extract semantic information, using 3D only as a medium to localize this information with respect to the end-effector. We show across 93 simulated and 6 real tasks that when trained end-to-end with a variety of IL algorithms, Adapt3R maintains these algorithms' learning capacity while enabling zero-shot transfer to novel embodiments and camera poses.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 58 of its 74 references that are in the map. First posted on arXiv 6 Mar 2025.

Builds on

58 of 74 references.

Closest ancestors · 6

Same direction, newest first

  1. 20243D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representations6 Mar
  2. 2023MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations26 Oct
  3. 2023BridgeData V2: A Dataset for Robot Learning at Scale24 Aug
  4. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware23 Apr
  5. 2020Learning to Generalize Across Long-Horizon Tasks from Human Demonstrations13 Mar
  6. 2019Self-Supervised Correspondence in Visuomotor Policy Learning16 Sep

Other references · 52

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  7. 2023GPT-4 Technical Reportcited by 18,132
  8. 2020Denoising Diffusion Implicit Modelscited by 9,655
  9. 2023Segment Anythingcited by 8,158
  10. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  11. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  12. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
38 more
  1. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  2. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  3. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  4. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  5. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  6. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  7. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  8. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  9. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
  10. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  11. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
  12. 2018On the Spectral Bias of Neural Networkscited by 1,321
  13. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 849
  14. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  15. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 801
  16. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  17. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 545
  18. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 484
  19. 2021Implicit Behavioral Cloningcited by 442
  20. 2024Robotic Control via Embodied Chain-of-Thought Reasoningcited by 340
  21. 2024TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulationcited by 331
  22. 2017Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperationcited by 286
  23. 20243D Diffuser Actor: Policy Diffusion with 3D Scene Representationscited by 278
  24. 2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 248
  25. 2023RVT: Robotic View Transformer for 3D Object Manipulationcited by 224
  26. 2025FoundationStereo: Zero-Shot Stereo Matchingcited by 168
  27. 2024Behavior Generation with Latent Actionscited by 159
  28. 2023Scaling Up and Distilling Down: Language-Guided Robot Skill Acquisitioncited by 156
  29. 2023Act3D: 3D Feature Field Transformers for Multi-Task Robotic Manipulationcited by 135
  30. 2024RVT-2: Learning Precise Manipulation from Few Demonstrationscited by 135
  31. 2023Language Embedded Radiance Fields for Zero-Shot Task-Oriented Graspingcited by 91
  32. 2024BAKU: An Efficient Transformer for Multi-Task Policy Learningcited by 80
  33. 2020Provable Benefit of Orthogonal Initialization in Optimizing Deep Linear Networkscited by 73
  34. 2024Diffusion-VLA: Generalizable and Interpretable Robot Foundation Model via Self-Generated Reasoningcited by 57
  35. 2024QueST: Self-Supervised Skill Abstractions for Learning Continuous Controlcited by 51
  36. 2024RoVi-Aug: Robot and Viewpoint Augmentation for Cross-Embodiment Robot Learningcited by 46
  37. 2023D3Fields: Dynamic 3D Descriptor Fields for Zero-Shot Generalizable Rearrangementcited by 35
  38. 2022Frame Mining: a Free Lunch for Learning Robotic Manipulation from 3D Point Cloudscited by 31

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 12.

16 papers.

Sort:

By area:

  1. 2025GeoAware-VLA: Implicit Geometry Aware Vision-Language-Action Modelcited by 24
  2. 2025GP3: A 3D Geometry-Aware Policy with Multi-View Images for Robotic Manipulationcited by 11
  3. 2025VLA Models Are More Generalizable Than You Think: Revisiting Physical and Spatial Modelingcited by 11
  4. 2026Geometric Action Model for Robot Policy Learningcited by 10
  5. 2026AnyCamVLA: Zero-Shot Camera Adaptation for Viewpoint Robust Vision-Language-Action Modelscited by 7
  6. 2026HeRO: Hierarchical 3D Semantic Representation for Pose-aware Object Manipulationcited by 3
  7. 2026Sim2real Image Translation Enables Viewpoint-Robust Policies from Fixed-Camera Datasetscited by 2
  8. 2026Fourier Features Let Agents Learn High Precision Policies with Imitation Learningcited by 2
  9. 2025Large Pre-Trained Models for Bimanual Manipulation in 3Dcited by 1
  10. 2026Fiducial Exoskeletons: Image-Centric Robot State Estimationcited by 1
  11. 2026See like a Robot: Robot-Centric Pointmaps for VLA Modelscited by 1
  12. 2026Point Cloud Sequence Encoding for Material-conditioned Graph Network Simulatorscited by 0
4 more
  1. 2026Domain Arithmetic: One-Shot VLA Adaptation under Environmental Shiftscited by 0
  2. 2026Learning 3D Affordances for Blade Insertion in Cluttered Stowingcited by 0
  3. 2026RayViT: Ray-Conditioned Visual Representations for Viewpoint-Robust Imitation Learningcited by 0
  4. 2026VersaCamVLA: Camera-Configurable VLA Policies for Robotic Manipulationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026R3D: Revisiting 3D Policy Learning
  2. 2022Practical Imitation Learning in the Real World via Task Consistency Loss
  3. 2026Human-to-Robot Interaction: Learning from Video Demonstration for Robot Imitation
  4. 2025Improving Robotic Manipulation with Efficient Geometry-Aware Vision Encoder
  5. 2026Robo3R: Enhancing Robotic Manipulation with Accurate Feed-Forward 3D Reconstruction
  6. 2026R2RDreamer: 3D-aware Data Augmentation for Spatially-generalized 2D Manipulation Policies
  7. 2025Canonical Policy: Learning Canonical 3D Representation for SE(3)-Equivariant Policy
  8. 2024RISE: 3D Perception Makes Real-World Robot Imitation Simple and Effective