# Meta-Learning Regrasping Strategies for Physical-Agnostic Objects

arXiv: https://arxiv.org/abs/2205.11110 · HTML: https://mlsanity.com/paper/2205.11110/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 59 references that are in the map.
- First posted on arXiv 23 May 2022.

## Builds on (most cited first, 30 of 33)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [Meta-Learning in Neural Networks: A Survey](https://mlsanity.com/paper/2004.05439/) (2020)
- [QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation](https://mlsanity.com/paper/1806.10293/) (2018)
- [Conditional Neural Processes](https://mlsanity.com/paper/1807.01613/) (2018)
- [Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection](https://mlsanity.com/paper/1603.02199/) (2016)
- [Neural Processes](https://mlsanity.com/paper/1807.01622/) (2018)
- [Attentive Neural Processes](https://mlsanity.com/paper/1901.05761/) (2019)
- [Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics](https://mlsanity.com/paper/1703.09312/) (2017)
- [Convolutional Conditional Neural Processes](https://mlsanity.com/paper/1910.13556/) (2019)
- [ACRONYM: A Large-Scale Grasp Dataset Based on Simulation](https://mlsanity.com/paper/2011.09584/) (2020)
- [Deep Learning for Detecting Robotic Grasps](https://mlsanity.com/paper/1301.3592/) (2013)
- [Meta-Learning Probabilistic Inference For Prediction](https://mlsanity.com/paper/1805.09921/) (2018)
- [Grasping in the Wild:Learning 6DoF Closed-Loop Grasping from Low-Cost Demonstrations](https://mlsanity.com/paper/1912.04344/) (2019)
- [Multimodal Model-Agnostic Meta-Learning via Task-Aware Modulation](https://mlsanity.com/paper/1910.13616/) (2019)
- [Volumetric Grasping Network: Real-time 6 DOF Grasp Detection in Clutter](https://mlsanity.com/paper/2101.01132/) (2021)
- [Jacquard: A Large Scale Dataset for Robotic Grasp Detection](https://mlsanity.com/paper/1803.11469/) (2018)
- [EGAD! an Evolved Grasping Analysis Dataset for diversity and reproducibility in robotic manipulation](https://mlsanity.com/paper/2003.01314/) (2020)
- [Improving Generalization in Meta-learning via Task Augmentation](https://mlsanity.com/paper/2007.13040/) (2020)
- [The Functional Neural Process](https://mlsanity.com/paper/1906.08324/) (2019)
- [Deep Reinforcement Learning for Vision-Based Robotic Grasping: A Simulated Comparative Evaluation of Off-Policy Methods](https://mlsanity.com/paper/1802.10264/) (2018)
- [Doubly Stochastic Variational Inference for Neural Processes with Hierarchical Latent Variables](https://mlsanity.com/paper/2008.09469/) (2020)
- [DemoGrasp: Few-Shot Learning for Robotic Grasping with Human Demonstration](https://mlsanity.com/paper/2112.02849/) (2021)
- [Function Contrastive Learning of Transferable Meta-Representations](https://mlsanity.com/paper/2010.07093/) (2020)
- [What Matters For Meta-Learning Vision Regression Tasks?](https://mlsanity.com/paper/2203.04905/) (2022)
- [MVGrasp: Real-Time Multi-View 3D Object Grasping in Highly Cluttered Environments](https://mlsanity.com/paper/2103.10997/) (2021)
- [DefGraspSim: Physics-based simulation of grasp outcomes for 3D deformable objects](https://mlsanity.com/paper/2203.11274/) (2022)
- [Contrastive Neural Processes for Self-Supervised Learning](https://mlsanity.com/paper/2110.13623/) (2021)
- [Robot Learning of 6 DoF Grasping using Model-based Adaptive Primitives](https://mlsanity.com/paper/2103.12810/) (2021)

## Built on by (most cited first, 7 of 7)

- [The Neural Process Family: Survey, Applications and Perspectives](https://mlsanity.com/paper/2209.00517/) (2022)
- [Language-guided Robot Grasping: CLIP-based Referring Grasp Synthesis in Clutter](https://mlsanity.com/paper/2311.05779/) (2023)
- [Memory Efficient Neural Processes via Constant Memory Attention Block](https://mlsanity.com/paper/2305.14567/) (2023)
- [Uncertainty-driven Exploration Strategies for Online Grasp Learning](https://mlsanity.com/paper/2309.12038/) (2023)
- [Category-Agnostic 6D Pose Estimation with Conditional Neural Processes](https://mlsanity.com/paper/2206.07162/) (2022)
- [Real-time Adaptation for Condition Monitoring Signal Prediction using Label-aware Neural Processes](https://mlsanity.com/paper/2403.16377/) (2024)
- [Towards Generalization on Real Domain for Single Image Dehazing via Meta-Learning](https://mlsanity.com/paper/2211.07147/) (2022)

## Abstract (arXiv)

Grasping inhomogeneous objects in real-world applications remains a challenging task due to the unknown physical properties such as mass distribution and coefficient of friction. In this study, we propose a meta-learning algorithm called ConDex, which incorporates Conditional Neural Processes (CNP) with DexNet-2.0 to autonomously discern the underlying physical properties of objects using depth images. ConDex efficiently acquires physical embeddings from limited trials, enabling precise grasping point estimation. Furthermore, ConDex is capable of updating the predicted grasping quality iteratively from new trials in an online fashion. To the best of our knowledge, we are the first who generate two object datasets focusing on inhomogeneous physical properties with varying mass distributions and friction coefficients. Extensive evaluations in simulation demonstrate ConDex's superior performance over DexNet-2.0 and existing meta-learning-based grasping pipelines. Furthermore, ConDex shows robust generalization to previously unseen real-world objects despite training solely in the simulation. The synthetic and real-world datasets will be published as well.