# ProgressLabeller: Visual Data Stream Annotation for Training Object-Centric 3D Perception

arXiv: https://arxiv.org/abs/2203.00283 · HTML: https://mlsanity.com/paper/2203.00283/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 35 references that are in the map.
- First posted on arXiv 1 Mar 2022.

## Builds on (most cited first, 20 of 20)

- [Neural Sparse Voxel Fields](https://mlsanity.com/paper/2007.11571/) (2020)
- [ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM](https://mlsanity.com/paper/2007.11898/) (2020)
- [ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras](https://mlsanity.com/paper/1610.06475/) (2016)
- [PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes](https://mlsanity.com/paper/1711.00199/) (2017)
- [CosyPose: Consistent multi-view multi-object 6D pose estimation](https://mlsanity.com/paper/2008.08465/) (2020)
- [T-LESS: An RGB-D Dataset for 6D Pose Estimation of Texture-less Objects](https://mlsanity.com/paper/1701.05498/) (2017)
- [BOP Challenge 2020 on 6D Object Localization](https://mlsanity.com/paper/2009.07378/) (2020)
- [Category-Level Articulated Object Pose Estimation](https://mlsanity.com/paper/1912.11913/) (2019)
- [Objectron: A Large Scale Dataset of Object-Centric Videos in the Wild with Pose Annotations](https://mlsanity.com/paper/2012.09988/) (2020)
- [FFB6D: A Full Flow Bidirectional Fusion Network for 6D Pose Estimation](https://mlsanity.com/paper/2103.02242/) (2021)
- [Deep Extreme Cut: From Extreme Points to Object Segmentation](https://mlsanity.com/paper/1711.09081/) (2017)
- [LatentFusion: End-to-End Differentiable Reconstruction and Rendering for Unseen Object Pose Estimation](https://mlsanity.com/paper/1912.00416/) (2019)
- [BlenderProc](https://mlsanity.com/paper/1911.01911/) (2019)
- [KeyPose: Multi-View 3D Labeling and Keypoint Estimation for Transparent Objects](https://mlsanity.com/paper/1912.02805/) (2019)
- [EasyLabel: A Semi-Automatic Pixel-wise Object Annotation Tool for Creating Robotic RGB-D Datasets](https://mlsanity.com/paper/1902.01626/) (2019)
- [StereOBJ-1M: Large-scale Stereo Image Dataset for 6D Object Pose Estimation](https://mlsanity.com/paper/2109.10115/) (2021)
- [LabelFusion: A Pipeline for Generating Ground Truth Labels for Real RGBD Data of Cluttered Scenes](https://mlsanity.com/paper/1707.04796/) (2017)
- [SALT: A Semi-automatic Labeling Tool for RGB-D Video Sequences](https://mlsanity.com/paper/2102.10820/) (2021)
- [GRIP: Generative Robust Inference and Perception for Semantic Robot Manipulation in Adversarial Environments](https://mlsanity.com/paper/1903.08352/) (2019)
- [Rapid Pose Label Generation through Sparse Representation of Unknown Objects](https://mlsanity.com/paper/2011.03790/) (2020)

## Built on by (most cited first, 6 of 6)

- [ClearPose: Large-scale Transparent Object Dataset and Benchmark](https://mlsanity.com/paper/2203.03890/) (2022)
- [TRansPose: Large-Scale Multispectral Dataset for Transparent Object](https://mlsanity.com/paper/2307.05016/) (2023)
- [FDCT: Fast Depth Completion for Transparent Objects](https://mlsanity.com/paper/2307.12274/) (2023)
- [NeuralLabeling: A versatile toolset for labeling vision datasets using Neural Radiance Fields](https://mlsanity.com/paper/2309.11966/) (2023)
- [SPLATART: Articulated Gaussian Splatting with Estimated Object Structure](https://mlsanity.com/paper/2506.12184/) (2025)
- [TransBiolab: A Real-World Multi-View Dataset of Cluttered Transparent Biomedical Objects](https://mlsanity.com/paper/2607.21071/) (2026)

## Abstract (arXiv)

Visual perception tasks often require vast amounts of labelled data, including 3D poses and image space segmentation masks. The process of creating such training data sets can prove difficult or time-intensive to scale up to efficacy for general use. Consider the task of pose estimation for rigid objects. Deep neural network based approaches have shown good performance when trained on large, public datasets. However, adapting these networks for other novel objects, or fine-tuning existing models for different environments, requires significant time investment to generate newly labelled instances. Towards this end, we propose ProgressLabeller as a method for more efficiently generating large amounts of 6D pose training data from color images sequences for custom scenes in a scalable manner. ProgressLabeller is intended to also support transparent or translucent objects, for which the previous methods based on depth dense reconstruction will fail. We demonstrate the effectiveness of ProgressLabeller by rapidly create a dataset of over 1M samples with which we fine-tune a state-of-the-art pose estimation network in order to markedly improve the downstream robotic grasp success rates. ProgressLabeller is open-source at https://github.com/huijieZH/ProgressLabeller.