mlsanityData to 14 Oct 2026

Equivariant Descriptor Fields: SE(3)-Equivariant Energy-Based Models for End-to-End Visual Robotic Manipulation Learning

arXiv 2206.08321 ↗First posted 16 Jun 2022cs.RO cs.AIcited by 56

Not on the map. Why?

Abstract (arXiv)

End-to-end learning for visual robotic manipulation is known to suffer from sample inefficiency, requiring large numbers of demonstrations. The spatial roto-translation equivariance, or the SE(3)-equivariance can be exploited to improve the sample efficiency for learning robotic manipulation. In this paper, we present SE(3)-equivariant models for visual robotic manipulation from point clouds that can be trained fully end-to-end. By utilizing the representation theory of the Lie group, we construct novel SE(3)-equivariant energy-based models that allow highly sample efficient end-to-end learning. We show that our models can learn from scratch without prior knowledge and yet are highly sample efficient (5~10 demonstrations are enough). Furthermore, we show that our models can generalize to tasks with (i) previously unseen target object poses, (ii) previously unseen target object instances of the category, and (iii) previously unseen visual distractors. We experiment with 6-DoF robotic manipulation tasks to validate our models' sample efficiency and generalizability. Codes are available at: https://github.com/tomato1mule/edf

Key facts. Cited by 56 papers in the map (data through 14 Oct 2026). Builds on 36 of its 85 references that are in the map. First posted on arXiv 16 Jun 2022.

Builds on

36 of 85 references.

References · 36

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2013Auto-Encoding Variational Bayescited by 4,053
  3. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,223
  4. 2018Relational inductive biases, deep learning, and graph networkscited by 1,360
  5. 2016Group Equivariant Convolutional Networkscited by 1,165
  6. 2015End-to-End Training of Deep Visuomotor Policiescited by 875
  7. 2018Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point cloudscited by 746
  8. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
  9. 2020SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networkscited by 607
  10. 2016Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithmcited by 507
  11. 2021Implicit Behavioral Cloningcited by 452
  12. 2020Transporter Networks: Rearranging the Visual World for Robotic Manipulationcited by 344
24 more
  1. 2020Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloudcited by 335
  2. 2021Vector Neurons: A General Framework for SO(3)-Equivariant Networkscited by 305
  3. 2021Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulationcited by 209
  4. 2016A Theory of Generative ConvNetcited by 174
  5. 2018Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulationcited by 142
  6. 2019Unsupervised Learning of Object Keypoints for Perception and Controlcited by 109
  7. 2019Self-supervised 6D Object Pose Estimation for Robot Manipulationcited by 101
  8. 2022SO(2)-Equivariant Reinforcement Learningcited by 84
  9. 2021Independent SE(3)-Equivariant Models for End-to-End Rigid Protein Dockingcited by 82
  10. 2020Learning to Rearrange Deformable Cables, Fabrics, and Bags with Goal-Conditioned Transporter Networkscited by 78
  11. 2016Multi-view Self-supervised Deep Learning for 6D Pose Estimation in the Amazon Picking Challengecited by 70
  12. 2018RGCNN: Regularized Graph CNN for Point Cloud Segmentationcited by 70
  13. 2016Cooperative Training of Descriptor and Generator Networkscited by 60
  14. 2020Generative PointNet: Deep Energy-Based Learning on Unordered Point Sets for 3D Generation, Reconstruction and Classificationcited by 57
  15. 2018Learning Descriptor Networks for 3D Shape Synthesis and Analysiscited by 56
  16. 2022Equivariant Transporter Networkcited by 46
  17. 2021Leveraging SE(3) Equivariance for Self-Supervised Category-Level Object Pose Estimationcited by 45
  18. 2022A Tale of Two Flows: Cooperative Learning of Langevin Flow and Normalizing Flow Toward Energy-Based Modelcited by 44
  19. 2020Learning Energy-Based Model with Variational Auto-Encoder as Amortized Samplercited by 43
  20. 2022Convergence of the Riemannian Langevin Algorithmcited by 25
  21. 2016Learning to Draw Samples: With Application to Amortized MLE for Generative Adversarial Learningcited by 21
  22. 2019Energy-Based Continuous Inverse Optimal Controlcited by 14
  23. 2021Learning Equivariant Energy Based Models with Equivariant Stein Variational Gradient Descentcited by 10
  24. 2021Hierarchical Primitive Composition: Simultaneous Activation of Skills with Inconsistent Action Dimensions in Multiple Hierarchiescited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 13; 2024: 19; 2025: 16; 2026: 7.

56 papers.

Sort:

By area:

  1. 2023Learning to Act from Actionless Videos through Dense Correspondencescited by 164
  2. 2024EquiBot: SIM(3)-Equivariant Diffusion Policy for Generalizable and Data Efficient Learningcited by 75
  3. 2024Equivariant Diffusion Policycited by 66
  4. 2023Diffusion-EDFs: Bi-equivariant Denoising Generative Modeling on SE(3) for Visual Robotic Manipulationcited by 52
  5. 2022SE(3)-Equivariant Relational Rearrangement with Neural Descriptor Fieldscited by 51
  6. 2025Diffusion Models for Robotic Manipulation: A Surveycited by 41
  7. 2023EquivAct: SIM(3)-Equivariant Visuomotor Policies beyond Rigid Object Manipulationcited by 37
  8. 2024A Survey of Embodied Learning for Object-Centric Robotic Manipulationcited by 35
  9. 2024Fourier Transporter: Bi-Equivariant Robotic Manipulation in 3Dcited by 33
  10. 2024HardNet: Hard-Constrained Neural Networks with Universal Approximation Guaranteescited by 31
  11. 2023Leveraging SE(3) Equivariance for Learning 3D Geometric Shape Assemblycited by 28
  12. 2024ActionFlow: Equivariant, Accurate, and Efficient Policies with Spatially Symmetric Flow Matchingcited by 26
38 more
  1. 2023SparseDFF: Sparse-View Feature Distillation for One-Shot Dexterous Manipulationcited by 22
  2. 2023Local Neural Descriptor Fields: Locally Conditioned Object Representations for Manipulationcited by 21
  3. 2023A General Theory of Correct, Incorrect, and Extrinsic Equivariancecited by 19
  4. 2023Banana: Banach Fixed-Point Network for Pointcloud Segmentation with Inter-Part Equivariancecited by 19
  5. 2023EFEM: Equivariant Neural Field Expectation Maximization for 3D Object Segmentation Without Scene Supervisioncited by 18
  6. 2024Imagination Policy: Using Generative Point Cloud Models for Learning Manipulation Policiescited by 18
  7. 2023One-shot Imitation Learning via Interaction Warpingcited by 16
  8. 2025A Survey of Robotic Navigation and Manipulation with Physics Simulators in the Era of Embodied AIcited by 16
  9. 2024Leveraging Locality to Boost Sample Efficiency in Robotic Manipulationcited by 13
  10. 2024RiEMann: Near Real-Time SE(3)-Equivariant Robot Manipulation without Point Cloud Segmentationcited by 12
  11. 2025AnyPlace: Learning Generalized Object Placement for Robot Manipulationcited by 11
  12. 2023NeuSE: Neural SE(3)-Equivariant Embedding for Long-Term Object-Based Simultaneous Localization and Mappingcited by 10
  13. 2025SE(3)-Equivariant Robot Learning and Control: A Tutorial Surveycited by 10
  14. 2023Denoising Heat-inspired Diffusion with Insulators for Collision Free Motion Planningcited by 8
  15. 2024EqvAfford: SE(3) Equivariance for Point-Level Affordance Learningcited by 6
  16. 2024MATCH POLICY: A Simple Pipeline from Point Cloud Registration to Manipulation Policiescited by 6
  17. 2024SE(3) Equivariant Ray Embeddings for Implicit Multi-View Depth Estimationcited by 6
  18. 2025Constraint-Preserving Data Generation for Visuomotor Policy Learningcited by 6
  19. 2023Unified framework for diffusion generative models in SO(3): applications in computer vision and astrophysicscited by 5
  20. 2024Multi-task real-robot data with gaze attention for dual-arm fine manipulationcited by 5
  21. 2024Equivariant Offline Reinforcement Learningcited by 5
  22. 2024Neural Attention Field: Emerging Point Relevance in 3D Scenes for One-Shot Dexterous Graspingcited by 5
  23. 2025A Practical Guide for Incorporating Symmetry in Diffusion Policycited by 5
  24. 2024Constrained 6-DoF Grasp Generation on Complex Shapes for Improved Dual-Arm Manipulationcited by 4
  25. 2024SE(3)-bi-equivariant Transformers for Point Cloud Assemblycited by 4
  26. 2025ADPro: a Test-time Adaptive Diffusion Policy via Manifold-constrained Denoising and Task-aware Initialization for Robotic Manipulationcited by 4
  27. 2025EfficientFlow: Efficient Equivariant Flow Policy Learning for Embodied AIcited by 4
  28. 2025Partially Equivariant Reinforcement Learning in Symmetry-Breaking Environmentscited by 3
  29. 2025MatrixNet: Learning over symmetry groups using learned group representationscited by 2
  30. 2025Coarse-to-Fine 3D Keyframe Transportercited by 2
  31. 2025Equivariant Flow Matching for Point Cloud Assemblycited by 2
  32. 2025Enforcing convex constraints in Graph Neural Networkscited by 2
  33. 2024BiEquiFormer: Bi-Equivariant Representations for Global Point Cloud Registrationcited by 1
  34. 2025Equivariant Volumetric Graspingcited by 1
  35. 2025Symmetry-Aware Steering of Equivariant Diffusion Policies: Benefits and Limitscited by 1
  36. 2024SPLIT: SE(3)-diffusion via Local Geometry-based Score Prediction for 3D Scene-to-Pose-Set Matching Problemscited by 0
  37. 2025Multi-Group Equivariant Augmentation for Reinforcement Learning in Robot Manipulationcited by 0
  38. 2026Group-Invariant Unsupervised Skill Discovery: Symmetry-aware Skill Representations for Generalizable Behaviorcited by 0

50 most cited of 56 All 56 (JSON)