# LensDFF: Language-enhanced Sparse Feature Distillation for Efficient Few-Shot Dexterous Manipulation

arXiv: https://arxiv.org/abs/2503.03890 · HTML: https://mlsanity.com/paper/2503.03890/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Multimodal distillation, one of 12 directions in Knowledge distillation.
- Builds on 26 of its 39 references that are in the map.
- First posted on arXiv 5 Mar 2025.

## Placed in

- Knowledge distillation / [Multimodal distillation](https://mlsanity.com/distill/multimodal-distillation/)

## Builds on (most cited first, 26 of 26)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [RegionCLIP: Region-based Language-Image Pretraining](https://mlsanity.com/paper/2112.09106/) (2021)
- [LERF: Language Embedded Radiance Fields](https://mlsanity.com/paper/2303.09553/) (2023)
- [FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects](https://mlsanity.com/paper/2312.08344/) (2023)
- [Decomposing NeRF for Editing via Feature Field Distillation](https://mlsanity.com/paper/2205.15585/) (2022)
- [Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulation](https://mlsanity.com/paper/2112.05124/) (2021)
- [DexGraspNet: A Large-Scale Robotic Dexterous Grasp Dataset for General Objects Based on Simulation](https://mlsanity.com/paper/2210.02697/) (2022)
- [Distilled Feature Fields Enable Few-Shot Language-Guided Manipulation](https://mlsanity.com/paper/2308.07931/) (2023)
- [Language Embedded Radiance Fields for Zero-Shot Task-Oriented Grasping](https://mlsanity.com/paper/2309.07970/) (2023)
- [GaussianGrasper: 3D Language Gaussian Splatting for Open-vocabulary Robotic Grasping](https://mlsanity.com/paper/2403.09637/) (2024)
- [GraspNeRF: Multiview-based 6-DoF Grasp Detection for Transparent and Specular Objects Using Generalizable NeRF](https://mlsanity.com/paper/2210.06575/) (2022)
- [DexDiffuser: Generating Dexterous Grasps with Diffusion Models](https://mlsanity.com/paper/2402.02989/) (2024)
- [DexGraspNet 2.0: Learning Generative Dexterous Grasping in Large-scale Synthetic Cluttered Scenes](https://mlsanity.com/paper/2410.23004/) (2024)
- [GraspSplats: Efficient Manipulation with 3D Feature Splatting](https://mlsanity.com/paper/2409.02084/) (2024)
- [Deep Differentiable Grasp Planner for High-DOF Grippers](https://mlsanity.com/paper/2002.01530/) (2020)
- [Learning Continuous 3D Reconstructions for Geometrically Aware Grasping](https://mlsanity.com/paper/1910.00983/) (2019)
- [Learning Robust Real-World Dexterous Grasping Policies via Implicit Shape Augmentation](https://mlsanity.com/paper/2210.13638/) (2022)
- [SparseDFF: Sparse-View Feature Distillation for One-Shot Dexterous Manipulation](https://mlsanity.com/paper/2310.16838/) (2023)
- [Planning Multi-Fingered Grasps as Probabilistic Inference in a Learned Deep Network](https://mlsanity.com/paper/1804.03289/) (2018)
- [MultiGripperGrasp: A Dataset for Robotic Grasping from Parallel Jaw Grippers to Dexterous Hands](https://mlsanity.com/paper/2403.09841/) (2024)
- [Neural Attention Field: Emerging Point Relevance in 3D Scenes for One-Shot Dexterous Grasping](https://mlsanity.com/paper/2410.23039/) (2024)
- [DexGANGrasp: Dexterous Generative Adversarial Grasping Synthesis for Task-Oriented Manipulation](https://mlsanity.com/paper/2407.17348/) (2024)

## Abstract (arXiv)

Learning dexterous manipulation from few-shot demonstrations is a significant yet challenging problem for advanced, human-like robotic systems. Dense distilled feature fields have addressed this challenge by distilling rich semantic features from 2D visual foundation models into the 3D domain. However, their reliance on neural rendering models such as Neural Radiance Fields (NeRF) or Gaussian Splatting results in high computational costs. In contrast, previous approaches based on sparse feature fields either suffer from inefficiencies due to multi-view dependencies and extensive training or lack sufficient grasp dexterity. To overcome these limitations, we propose Language-ENhanced Sparse Distilled Feature Field (LensDFF), which efficiently distills view-consistent 2D features onto 3D points using our novel language-enhanced feature fusion strategy, thereby enabling single-view few-shot generalization. Based on LensDFF, we further introduce a few-shot dexterous manipulation framework that integrates grasp primitives into the demonstrations to generate stable and highly dexterous grasps. Moreover, we present a real2sim grasp evaluation pipeline for efficient grasp assessment and hyperparameter tuning. Through extensive simulation experiments based on the real2sim pipeline and real-world experiments, our approach achieves competitive grasping performance, outperforming state-of-the-art approaches.