mlsanityData to 14 Oct 2026

SparseDFF: Sparse-View Feature Distillation for One-Shot Dexterous Manipulation

arXiv 2310.16838 ↗First posted 25 Oct 2023cs.RO cs.CVcited by 22

On the map: Knowledge distillation → Multimodal distillation

Abstract (arXiv)

Humans demonstrate remarkable skill in transferring manipulation abilities across objects of varying shapes, poses, and appearances, a capability rooted in their understanding of semantic correspondences between different instances. To equip robots with a similar high-level comprehension, we present SparseDFF, a novel DFF for 3D scenes utilizing large 2D vision models to extract semantic features from sparse RGBD images, a domain where research is limited despite its relevance to many tasks with fixed-camera setups. SparseDFF generates view-consistent 3D DFFs, enabling efficient one-shot learning of dexterous manipulations by mapping image features to a 3D point cloud. Central to SparseDFF is a feature refinement network, optimized with a contrastive loss between views and a point-pruning mechanism for feature continuity. This facilitates the minimization of feature discrepancies w.r.t. end-effector parameters, bridging demonstrations and target manipulations. Validated in real-world scenarios with a dexterous hand, SparseDFF proves effective in manipulating both rigid and deformable objects, demonstrating significant generalization capabilities across object and scene variations.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Placed in Multimodal distillation, one of 12 directions in Knowledge distillation. Builds on 56 of its 73 references that are in the map. First posted on arXiv 25 Oct 2023.

Builds on

56 of 73 references.

References · 56

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  5. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  6. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 715
  7. 2019Deep Hough Voting for 3D Object Detection in Point Cloudscited by 648
  8. 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 559
  9. 2023LERF: Language Embedded Radiance Fieldscited by 543
  10. 2020PointContrast: Unsupervised Pre-training for 3D Point Cloud Understandingcited by 504
  11. 2021In-Place Scene Labelling and Understanding with Implicit Scene Representationcited by 415
  12. 2022Decomposing NeRF for Editing via Feature Field Distillationcited by 371
38 more
  1. 2022HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interactioncited by 327
  2. 2021Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulationcited by 209
  3. 2019Deep Dynamics Models for Learning Dexterous Manipulationcited by 206
  4. 2022Neural Feature Fusion Fields: 3D Distillation of Self-Supervised 2D Image Representationscited by 196
  5. 2020Grasping Field: Learning Implicit Representations for Human Graspscited by 179
  6. 2022Panoptic Lifting for 3D Scene Understanding with Neural Fieldscited by 174
  7. 2021A System for General In-Hand Object Re-Orientationcited by 165
  8. 2022CLIP-Fields: Weakly Supervised Semantic Fields for Robotic Memorycited by 165
  9. 2019kPAM: KeyPoint Affordances for Category-Level Robotic Manipulationcited by 153
  10. 2018Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulationcited by 142
  11. 2022DexGraspNet: A Large-Scale Robotic Dexterous Grasp Dataset for General Objects Based on Simulationcited by 137
  12. 2022SE(3)-DiffusionFields: Learning smooth cost functions for joint grasp and motion optimization through diffusioncited by 127
  13. 2023UniDexGrasp: Universal Robotic Dexterous Grasping via Learning Diverse Proposal Generation and Goal-Conditioned Policycited by 124
  14. 2022In-Hand Object Rotation via Rapid Motor Adaptationcited by 118
  15. 2021D-Grasp: Physically Plausible Dynamic Grasp Synthesis for Hand-Object Interactionscited by 113
  16. 2023UniDexGrasp++: Improving Dexterous Grasping Policy Learning via Geometry-aware Curriculum and Iterative Generalist-Specialist Learningcited by 107
  17. 2015Benchmarking in Manipulation Research: The YCB Object and Model Set and Benchmarking Protocolscited by 99
  18. 2023GNFactor: Multi-Task Real Robot Learning with Generalizable Neural Feature Fieldscited by 99
  19. 2023Language Embedded Radiance Fields for Zero-Shot Task-Oriented Graspingcited by 91
  20. 2022DexVIP: Learning Dexterous Grasping with Human Hand Pose Priors from Videocited by 89
  21. 2020Learning Dexterous Grasping with Object-Centric Visual Affordancescited by 85
  22. 2022DexPoint: Generalizable Point Cloud Reinforcement Learning for Sim-to-Real Dexterous Manipulationcited by 84
  23. 2021VAT-Mart: Learning Visual Action Trajectory Proposals for Manipulating 3D ARTiculated Objectscited by 80
  24. 2022GraspNeRF: Multiview-based 6-DoF Grasp Detection for Transparent and Specular Objects Using Generalizable NeRFcited by 67
  25. 2020Dark, Beyond Deep: A Paradigm Shift to Cognitive AI with Humanlike Common Sensecited by 62
  26. 2022Neural Volumetric Object Selectioncited by 61
  27. 2022Equivariant Descriptor Fields: SE(3)-Equivariant Energy-Based Models for End-to-End Visual Robotic Manipulation Learningcited by 56
  28. 2022GenDexGrasp: Generalizable Dexterous Graspingcited by 56
  29. 2022Learning Generalizable Dexterous Manipulation from Human Grasp Affordancecited by 51
  30. 2022SE(3)-Equivariant Relational Rearrangement with Neural Descriptor Fieldscited by 51
  31. 2021Generalization in Dexterous Manipulation via Geometry-Aware Multi-Task Learningcited by 50
  32. 2016Learning Dexterous Manipulation Policies from Experience and Imitationcited by 33
  33. 2022Neural Grasp Distance Fields for Robot Manipulationcited by 31
  34. 2022USEEK: Unsupervised SE(3)-Equivariant 3D Keypoints for Generalizable Manipulationcited by 27
  35. 2023Learning Foresightful Dense Visual Affordance for Deformable Object Manipulationcited by 25
  36. 2022Learning High-DOF Reaching-and-Grasping via Dynamic Representation of Gripper-Object Interactioncited by 24
  37. 2023SpawnNet: Learning Generalizable Visuomotor Skills from Pre-trained Networkscited by 24
  38. 2018Planning Multi-Fingered Grasps as Probabilistic Inference in a Learned Deep Networkcited by 19

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 8; 2025: 8; 2026: 5.

22 papers.

Sort:

By area:

  1. 2023ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic Manipulationcited by 162
  2. 2024RoboMamba: Efficient Vision-Language-Action Model for Robotic Reasoning and Manipulationcited by 103
  3. 2025DexGraspVLA: A Vision-Language-Action Framework Towards General Dexterous Graspingcited by 89
  4. 2024GaussianGrasper: 3D Language Gaussian Splatting for Open-vocabulary Robotic Graspingcited by 78
  5. 2025Motion Tracks: A Unified Representation for Human-Robot Transfer in Few-Shot Imitation Learningcited by 59
  6. 2024Lift3D Foundation Policy: Lifting 2D Large-Scale Pretrained Models for Robust 3D Robotic Manipulationcited by 46
  7. 2024DINOBot: Robot Manipulation via Retrieval and Alignment with Vision Foundation Modelscited by 44
  8. 2025VLA-R1: Enhancing Reasoning in Vision-Language-Action Modelscited by 38
  9. 2025ManipLVM-R1: Reinforcement Learning for Reasoning in Embodied Manipulation with Large Vision-Language Modelscited by 27
  10. 2024GarmentLab: A Unified Simulation and Benchmark for Garment Manipulationcited by 22
  11. 2024A Self-Correcting Vision-Language-Action Model for Fast and Slow System Manipulationcited by 21
  12. 2024Multiview Equivariance Improves 3D Correspondence Understanding with Minimal Feature Finetuningcited by 14
10 more
  1. 2024Neural Attention Field: Emerging Point Relevance in 3D Scenes for One-Shot Dexterous Graspingcited by 5
  2. 2026Learning Part-Aware Dense 3D Feature Field for Generalizable Articulated Object Manipulationcited by 5
  3. 2025Learning 3D Scene Analogies with Neural Contextual Scene Mapscited by 4
  4. 2025Towards Fast, Memory-based and Data-Efficient Vision-Language Policycited by 2
  5. 2025Point2Act: Efficient 3D Distillation of Multimodal LLMs for Zero-Shot Context-Aware Graspingcited by 2
  6. 2026Generate, Transfer, Adapt: Learning Functional Dexterous Grasping from a Single Human Demonstrationcited by 1
  7. 2026Analogical Trajectory Transfercited by 1
  8. 2025LensDFF: Language-enhanced Sparse Feature Distillation for Efficient Few-Shot Dexterous Manipulationcited by 0
  9. 2026Flow-Enabled Generalization to Human Demonstrations in Few-Shot Imitation Learningcited by 0
  10. 2026Tele-Catch: Adaptive Teleoperation for Dexterous Dynamic 3D Object Catchingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Dexterity-BEV: Aligning 3D World and Actions for Generalizable Robot Policies Learning
  2. 2026ZeroDex: Zero-Shot Long-Horizon Dexterous Manipulation via Multi-View 3D-Grounded VLM Reasoning
  3. 2023Distilled Feature Fields Enable Few-Shot Language-Guided Manipulation
  4. 2026AdvDex: Learning Dexterous Manipulation from Human Demonstrations via Joint-Aligned Actions and Adversarial Learning
  5. 2024Click to Grasp: Zero-Shot Precise Manipulation via Visual Diffusion Descriptors
  6. 2026AnyViewDex: View-Invariant Dexterous Manipulation from RGB Observations ↗
  7. 2024Learning Generalizable Feature Fields for Mobile Manipulation
  8. 2024UniAff: A Unified Representation of Affordances for Tool Usage and Articulation with Vision-Language Models