# Remote Task-oriented Grasp Area Teaching By Non-Experts through Interactive Segmentation and Few-Shot Learning

arXiv: https://arxiv.org/abs/2303.10195 · HTML: https://mlsanity.com/paper/2303.10195/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 22 references that are in the map.
- First posted on arXiv 17 Mar 2023.

## Builds on (most cited first, 14 of 14)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [6-DOF GraspNet: Variational Grasp Generation for Object Manipulation](https://mlsanity.com/paper/1905.10520/) (2019)
- [Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenes](https://mlsanity.com/paper/2103.14127/) (2021)
- [AffordanceNet: An End-to-End Deep Learning Approach for Object Affordance Detection](https://mlsanity.com/paper/1709.07326/) (2017)
- [f-BRS: Rethinking Backpropagating Refinement for Interactive Segmentation](https://mlsanity.com/paper/2001.10331/) (2020)
- [Reducing the Barrier to Entry of Complex Robotic Software: a MoveIt! Case Study](https://mlsanity.com/paper/1404.3785/) (2014)
- [PointNetGPD: Detecting Grasp Configurations from Point Sets](https://mlsanity.com/paper/1809.06267/) (2018)
- [DemoGrasp: Few-Shot Learning for Robotic Grasping with Human Demonstration](https://mlsanity.com/paper/2112.02849/) (2021)
- [CAGE: Context-Aware Grasping Engine](https://mlsanity.com/paper/1909.11142/) (2019)
- [Attribute-Based Robotic Grasping with One-Grasp Adaptation](https://mlsanity.com/paper/2104.02271/) (2021)
- [Learning to Grasp from a Single Demonstration](https://mlsanity.com/paper/1806.03486/) (2018)
- [Task-grasping from human demonstration](https://mlsanity.com/paper/2203.00733/) (2022)

## Built on by (most cited first, 1 of 1)

- [A Prompt-driven Task Planning Method for Multi-drones based on Large Language Model](https://mlsanity.com/paper/2406.00006/) (2024)

## Abstract (arXiv)

A robot operating in unstructured environments must be able to discriminate between different grasping styles depending on the prospective manipulation task. Having a system that allows learning from remote non-expert demonstrations can very feasibly extend the cognitive skills of a robot for task-oriented grasping. We propose a novel two-step framework towards this aim. The first step involves grasp area estimation by segmentation. We receive grasp area demonstrations for a new task via interactive segmentation, and learn from these few demonstrations to estimate the required grasp area on an unseen scene for the given task. The second step is autonomous grasp estimation in the segmented region. To train the segmentation network for few-shot learning, we built a grasp area segmentation (GAS) dataset with 10089 images grouped into 1121 segmentation tasks. We benefit from an efficient meta learning algorithm for training for few-shot adaptation. Experimental evaluation showed that our method successfully detects the correct grasp area on the respective objects in unseen test scenes and effectively allows remote teaching of new grasp strategies by non-experts.