# Semi-Automatic Labeling for Deep Learning in Robotics

arXiv: https://arxiv.org/abs/1908.01862 · HTML: https://mlsanity.com/paper/1908.01862/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Aug 2019.

## Built on by (most cited first, 8 of 8)

- [Advancing 3D Point Cloud Understanding through Deep Transfer Learning: A Comprehensive Survey](https://mlsanity.com/paper/2407.17877/) (2024)
- [Automatically Prepare Training Data for YOLO Using Robotic In-Hand Observation and Synthesis](https://mlsanity.com/paper/2301.01441/) (2023)
- [Robotic Waste Sorter with Agile Manipulation and Quickly Trainable Detector](https://mlsanity.com/paper/2104.01260/) (2021)
- [FARPLS: A Feature-Augmented Robot Trajectory Preference Labeling System to Assist Human Labelers' Preference Elicitation](https://mlsanity.com/paper/2403.06267/) (2024)
- [A YOLO-Based Semi-Automated Labeling Approach to Improve Fault Detection Efficiency in Railroad Videos](https://mlsanity.com/paper/2504.01010/) (2025)
- [DeepScanner: a Robotic System for Automated 2D Object Dataset Collection with Annotations](https://mlsanity.com/paper/2108.02555/) (2021)
- [Efficiently Collecting Training Dataset for 2D Object Detection by Online Visual Feedback](https://mlsanity.com/paper/2304.04901/) (2023)
- [Feedback-driven object detection and iterative model improvement](https://mlsanity.com/paper/2411.19835/) (2024)

## Abstract (arXiv)

In this paper, we propose Augmented Reality Semi-automatic labeling (ARS), a semi-automatic method which leverages on moving a 2D camera by means of a robot, proving precise camera tracking, and an augmented reality pen to define initial object bounding box, to create large labeled datasets with minimal human intervention. By removing the burden of generating annotated data from humans, we make the Deep Learning technique applied to computer vision, that typically requires very large datasets, truly automated and reliable. With the ARS pipeline, we created effortlessly two novel datasets, one on electromechanical components (industrial scenario) and one on fruits (daily-living scenario), and trained robustly two state-of-the-art object detectors, based on convolutional neural networks, such as YOLO and SSD. With respect to the conventional manual annotation of 1000 frames that takes us slightly more than 10 hours, the proposed approach based on ARS allows annotating 9 sequences of about 35000 frames in less than one hour, with a gain factor of about 450. Moreover, both the precision and recall of object detection is increased by about 15% with respect to manual labeling. All our software is available as a ROS package in a public repository alongside the novel annotated datasets.