mlsanityData to 7 Oct 2026

Local Neural Descriptor Fields: Locally Conditioned Object Representations for Manipulation

arXiv 2302.03573 ↗First posted 7 Feb 2023cs.RO cs.AI cs.CVcited by 21

Not on the map. Why?

Abstract (arXiv)

A robot operating in a household environment will see a wide range of unique and unfamiliar objects. While a system could train on many of these, it is infeasible to predict all the objects a robot will see. In this paper, we present a method to generalize object manipulation skills acquired from a limited number of demonstrations, to novel objects from unseen shape categories. Our approach, Local Neural Descriptor Fields (L-NDF), utilizes neural descriptors defined on the local geometry of the object to effectively transfer manipulation demonstrations to novel objects at test time. In doing so, we leverage the local geometry shared between objects to produce a more general manipulation framework. We illustrate the efficacy of our approach in manipulating novel objects in novel poses -- both in simulation and in the real world.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 23 of its 32 references that are in the map. First posted on arXiv 7 Feb 2023.

Builds on

23 of 32 references.

References · 23

  1. 2017Mask R-CNNcited by 7,259
  2. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  3. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  4. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  5. 2020Convolutional Occupancy Networkscited by 760
  6. 2021NICE-SLAM: Neural Implicit Scalable Encoding for SLAMcited by 586
  7. 2021iMAP: Implicit Mapping and Positioning in Real-Timecited by 476
  8. 2020INeRF: Inverting Neural Radiance Fields for Pose Estimationcited by 381
  9. 2021Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulationcited by 204
  10. 2021Vision-Only Robot Navigation in a Neural Radiance Worldcited by 153
  11. 2019kPAM: KeyPoint Affordances for Category-Level Robotic Manipulationcited by 152
  12. 2018Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulationcited by 140
11 more
  1. 2021Dex-NeRF: Using a Neural Radiance Field to Grasp Transparent Objectscited by 119
  2. 2021LENS: Localization enhanced by NeRF synthesiscited by 116
  3. 2021Synergies Between Affordance and Geometry: 6-DoF Grasp Detection via Implicit Representationscited by 107
  4. 2022iSDF: Real-Time Neural Signed Distance Fields for Robot Perceptioncited by 107
  5. 2022NeRF-Supervision: Learning Dense Object Descriptors from Neural Radiance Fieldscited by 86
  6. 2022Equivariant Descriptor Fields: SE(3)-Equivariant Energy-Based Models for End-to-End Visual Robotic Manipulation Learningcited by 54
  7. 2020Learning Rope Manipulation Policies Using Dense Object Descriptors Trained on Synthetic Depth Datacited by 51
  8. 2022SE(3)-Equivariant Relational Rearrangement with Neural Descriptor Fieldscited by 51
  9. 2021kPAM 2.0: Feedback Control for Category-Level Robotic Manipulationcited by 38
  10. 2019kPAM-SC: Generalizable Manipulation Planning using KeyPoint Affordance and Shape Completioncited by 27
  11. 2022Robust Change Detection Based on Neural Descriptor Fieldscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 12; 2025: 3; 2026: 1.

21 papers.

Sort:

By area:

  1. 2024ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulationcited by 305
  2. 2023Learning to Act from Actionless Videos through Dense Correspondencescited by 159
  3. 2023Diffusion-EDFs: Bi-equivariant Denoising Generative Modeling on SE(3) for Visual Robotic Manipulationcited by 51
  4. 2024A Survey of Embodied Learning for Object-Centric Robotic Manipulationcited by 34
  5. 2024Fourier Transporter: Bi-Equivariant Robotic Manipulation in 3Dcited by 32
  6. 2025UAD: Unsupervised Affordance Distillation for Generalization in Robotic Manipulationcited by 23
  7. 2024Neural Fields in Robotics: A Surveycited by 21
  8. 2024Composable Part-Based Manipulationcited by 17
  9. 2023One-shot Imitation Learning via Interaction Warpingcited by 16
  10. 2024Bi-KVIL: Keypoints-based Visual Imitation Learning of Bimanual Manipulation Taskscited by 16
  11. 2024NeRFs in Robotics: A Surveycited by 16
  12. 2024Leveraging Locality to Boost Sample Efficiency in Robotic Manipulationcited by 13
9 more
  1. 2024RiEMann: Near Real-Time SE(3)-Equivariant Robot Manipulation without Point Cloud Segmentationcited by 12
  2. 2023NeuSE: Neural SE(3)-Equivariant Embedding for Consistent Spatial Understanding with Objectscited by 10
  3. 2025AnyPlace: Learning Generalized Object Placement for Robot Manipulationcited by 10
  4. 2023NOD-TAMP: Generalizable Long-Horizon Planning with Neural Object Descriptorscited by 8
  5. 2024FuncGrasp: Learning Object-Centric Neural Grasp Functions from Single Annotated Example Objectcited by 6
  6. 2024Grasp, See, and Place: Efficient Unknown Object Rearrangement with Policy Structure Priorcited by 5
  7. 2024Implicit Contact Diffuser: Sequential Contact Reasoning with Latent Point Cloud Diffusioncited by 1
  8. 2025Multi-Group Equivariant Augmentation for Reinforcement Learning in Robot Manipulationcited by 0
  9. 2026LAMP: Lift Image-Editing as General 3D Priors for Open-world Manipulationcited by 0