mlsanityData to 7 Oct 2026

Coarse-to-Fine 3D Keyframe Transporter

arXiv 2502.01773 ↗First posted 3 Feb 2025cs.RO cs.CVcited by 2

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

Recent advances in Keyframe Imitation Learning (IL) have enabled learning-based agents to solve a diverse range of manipulation tasks. However, most approaches ignore the rich symmetries in the problem setting and, as a consequence, are sample-inefficient. This work identifies and utilizes the bi-equivariant symmetry within Keyframe IL to design a policy that generalizes to transformations of both the workspace and the objects grasped by the gripper. We make two main contributions: First, we analyze the bi-equivariance properties of the keyframe action scheme and propose a Keyframe Transporter derived from the Transporter Networks, which evaluates actions using cross-correlation between the features of the grasped object and the features of the scene. Second, we propose a computationally efficient coarse-to-fine SE(3) action evaluation scheme for reasoning the intertwined translation and rotation action. The resulting method outperforms strong Keyframe IL baselines by an average of >10% on a wide range of simulation tasks, and by an average of 55% in 4 physical experiments.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 37 of its 45 references that are in the map. First posted on arXiv 3 Feb 2025.

Builds on

37 of 45 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022SEIL: Simulation-augmented Equivariant Imitation Learning31 Oct

Other references · 36

  1. 2017Attention Is All You Needcited by 59,932
  2. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  3. 20163D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotationcited by 1,471
  4. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,186
  5. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  6. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
  7. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 545
  8. 2019General E(2)-Equivariant Steerable CNNscited by 441
  9. 2020Transporter Networks: Rearranging the Visual World for Robotic Manipulationcited by 342
  10. 2023RVT: Robotic View Transformer for 3D Object Manipulationcited by 224
  11. 2021Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulationcited by 204
  12. 2023Act3D: 3D Feature Field Transformers for Multi-Task Robotic Manipulationcited by 135
24 more
  1. 2017Robotic Pick-and-Place of Novel Objects in Clutter with Multi-Affordance Grasping and Cross-Domain Image Matchingcited by 124
  2. 2021Coarse-to-Fine Q-attention: Efficient Learning for Visual Robotic Manipulation via Discretisationcited by 120
  3. 2018Closing the Loop for Robotic Grasping: A Real-time, Generative Grasp Synthesis Approachcited by 117
  4. 2024Hierarchical Diffusion Policy for Kinematics-Aware Multi-Task Robotic Manipulationcited by 92
  5. 2014Reducing the Barrier to Entry of Complex Robotic Software: a MoveIt! Case Studycited by 86
  6. 2021Q-attention: Enabling Efficient Learning for Vision-based Robotic Manipulationcited by 85
  7. 2022SO(2)-Equivariant Reinforcement Learningcited by 79
  8. 2021Equivariant Q Learning in Spatial Action Spacescited by 61
  9. 2024Equivariant Diffusion Policycited by 61
  10. 2022Equivariant Descriptor Fields: SE(3)-Equivariant Energy-Based Models for End-to-End Visual Robotic Manipulation Learningcited by 54
  11. 2022Sample Efficient Grasp Learning Using Equivariant Modelscited by 52
  12. 2023Diffusion-EDFs: Bi-equivariant Denoising Generative Modeling on SE(3) for Visual Robotic Manipulationcited by 51
  13. 2023cuRobo: Parallelized Collision-Free Minimum-Jerk Robot Motion Generationcited by 46
  14. 2022Equivariant Transporter Networkcited by 44
  15. 2022On-Robot Learning With Equivariant Modelscited by 38
  16. 2023DynPoint: Dynamic Neural Point For View Synthesiscited by 37
  17. 2022Edge Grasp Network: A Graph-Based SE(3)-invariant Approach to Grasp Detectioncited by 34
  18. 2024Fourier Transporter: Bi-Equivariant Robotic Manipulation in 3Dcited by 32
  19. 2022MIRA: Mental Imagery for Robotic Affordancescited by 26
  20. 2024Imagination Policy: Using Generative Point Cloud Models for Learning Manipulation Policiescited by 18
  21. 2024OrbitGrasp: SE(3)-Equivariant Grasp Learningcited by 15
  22. 2020Policy learning in SE(3) action spacescited by 14
  23. 2023On Robot Grasp Learning Using Equivariant Modelscited by 12
  24. 2019Learning Manipulation Skills Via Hierarchical Spatial Attentioncited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025SE(3)-Equivariant Diffusion Policy in Spherical Fourier Spacecited by 7
  2. 2026Improving Vision-Language-Action Model Fine-Tuning with Structured Stage and Keyframe Supervisioncited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2025Canonical Policy: Learning Canonical 3D Representation for SE(3)-Equivariant Policy
  2. 2026EquiBim: Learning Symmetry-Equivariant Policy for Bimanual Manipulation
  3. 2022K-VIL: Keypoints-based Visual Imitation Learning
  4. 2026KeyGen: Unsupervised Keypoint based Object-Centric Representations for Category-Level Policy Generalization
  5. 2026Pix2Act: Image-Space Manipulation Policies with Equivariant Augmentation
  6. 2021Keyframe-Focused Visual Imitation Learning
  7. 2026Enabling a Unified Cross-Domain Representation for Two-Finger Gripper Manipulation via Interaction-Centric Modeling
  8. 2026VolumeDP: Modeling Volumetric Representation for Manipulation Policy Learning