# The Collection of a Human Robot Collaboration Dataset for Cooperative Assembly in Glovebox Environments

arXiv: https://arxiv.org/abs/2407.14649 · HTML: https://mlsanity.com/paper/2407.14649/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 61 references that are in the map.
- First posted on arXiv 19 Jul 2024.

## Builds on (most cited first, 16 of 16)

- [RT-1: Robotics Transformer for Real-World Control at Scale](https://mlsanity.com/paper/2212.06817/) (2022)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Scaling Egocentric Vision: The EPIC-KITCHENS Dataset](https://mlsanity.com/paper/1804.02748/) (2018)
- [Faster Segment Anything: Towards Lightweight SAM for Mobile Applications](https://mlsanity.com/paper/2306.14289/) (2023)
- [FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Images](https://mlsanity.com/paper/1909.04349/) (2019)
- [First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotations](https://mlsanity.com/paper/1704.02463/) (2017)
- [The MECCANO Dataset: Understanding Human-Object Interactions from Egocentric Videos in an Industrial-like Domain](https://mlsanity.com/paper/2010.05654/) (2020)
- [In the Eye of the Beholder: Gaze and Actions in First Person Video](https://mlsanity.com/paper/2006.00626/) (2020)
- [BigHand2.2M Benchmark: Hand Pose Dataset and State of the Art Analysis](https://mlsanity.com/paper/1704.02612/) (2017)
- [Recurrent U-Net for Resource-Constrained Segmentation](https://mlsanity.com/paper/1906.04913/) (2019)
- [Analysis of Hand Segmentation in the Wild](https://mlsanity.com/paper/1803.03317/) (2018)
- [Assessing The Importance Of Colours For CNNs In Object Recognition](https://mlsanity.com/paper/2012.06917/) (2020)
- [Egocentric Video: A New Tool for Capturing Hand Use of Individuals with Spinal Cord Injury at Home](https://mlsanity.com/paper/1809.00928/) (2018)
- [Hand Segmentation for Hand-Object Interaction from Depth map](https://mlsanity.com/paper/1603.02345/) (2016)
- [Hand Action Detection from Ego-centric Depth Sequences with Error-correcting Hough Transform](https://mlsanity.com/paper/1606.02031/) (2016)
- [HaDR: Applying Domain Randomization for Generating Synthetic Multimodal Dataset for Hand Instance Segmentation in Cluttered Industrial Environments](https://mlsanity.com/paper/2304.05826/) (2023)

## Built on by (most cited first, 2 of 2)

- [Testing Human-Hand Segmentation on In-Distribution and Out-of-Distribution Data in Human-Robot Interactions Using a Deep Ensemble Model](https://mlsanity.com/paper/2501.07713/) (2025)
- [Train, Test, Re-evaluate: Schedule-Sensitive Evaluation of Generative Data for Hand Detection](https://mlsanity.com/paper/2606.01896/) (2026)

## Abstract (arXiv)

Industry 4.0 introduced AI as a transformative solution for modernizing manufacturing processes. Its successor, Industry 5.0, envisions humans as collaborators and experts guiding these AI-driven manufacturing solutions. Developing these techniques necessitates algorithms capable of safe, real-time identification of human positions in a scene, particularly their hands, during collaborative assembly. Although substantial efforts have curated datasets for hand segmentation, most focus on residential or commercial domains. Existing datasets targeting industrial settings predominantly rely on synthetic data, which we demonstrate does not effectively transfer to real-world operations. Moreover, these datasets lack uncertainty estimations critical for safe collaboration. Addressing these gaps, we present HAGS: Hand and Glove Segmentation Dataset. This dataset provides challenging examples to build applications toward hand and glove segmentation in industrial human-robot collaboration scenarios as well as assess out-of-distribution images, constructed via green screen augmentations, to determine ML-classifier robustness. We study state-of-the-art, real-time segmentation models to evaluate existing methods. Our dataset and baselines are publicly available.