# Train, Test, Re-evaluate: Schedule-Sensitive Evaluation of Generative Data for Hand Detection

arXiv: https://arxiv.org/abs/2606.01896 · HTML: https://mlsanity.com/paper/2606.01896/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 26 references that are in the map.
- First posted on arXiv 1 Jun 2026.

## Builds on (most cited first, 11 of 11)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [MediaPipe Hands: On-device Real-time Hand Tracking](https://mlsanity.com/paper/2006.10214/) (2020)
- [11K Hands: Gender recognition and biometric identification using a large dataset of hand images](https://mlsanity.com/paper/1711.04322/) (2017)
- [Towards Explaining Distribution Shifts](https://mlsanity.com/paper/2210.10275/) (2022)
- [A Simple Background Augmentation Method for Object Detection with Diffusion Model](https://mlsanity.com/paper/2408.00350/) (2024)
- [Analyzing the Synthetic-to-Real Domain Gap in 3D Hand Pose Estimation](https://mlsanity.com/paper/2503.19307/) (2025)
- [The Collection of a Human Robot Collaboration Dataset for Cooperative Assembly in Glovebox Environments](https://mlsanity.com/paper/2407.14649/) (2024)
- [AnyHand: A Large-Scale Synthetic Dataset for RGB(-D) Hand Pose Estimation](https://mlsanity.com/paper/2603.25726/) (2026)
- [Semantic Robustness Probing via Inpainting: An Interactive Tool for Safety-Critical Object Detection](https://mlsanity.com/paper/2605.27155/) (2026)

## Abstract (arXiv)

Generated (or synthetic) image data is increasingly used to augment or replace real training datasets when target imagery is scarce, expensive, or biased. For hand detection, particularly in occupational safety settings, public datasets mostly contain bare hands. This under-represents the variation in hand appearance introduced by gloves, tattoos, jewelry, and other personal protective equipment, creating a distribution shift that safety-critical applications encounter at deployment. We test whether generative inpainting, editing only the hand region of a real photograph to introduce accessories, can close this shift gap and improve detection of real hands at deployment. On a paired dataset of real images and their synthetic counterparts, we evaluate YOLOv8n hand detectors across six experiments (A-F), four of which involve training (A, C, D, E) under three random seeds each, evaluate them on a real test set and on a real-gloves-only test split, and report the mean average precision (mAP) at two overlap thresholds (mAP@0.5 and mAP@0.5:0.95) along with paired statistical tests. A two-stage experiment: train on real U synthetic data, then fine-tune the resulting weights on real-only at a lower learning rate, directionally improves mAP@0.5 compared to the real-only baseline model on the standard real test set, and narrows the real-gloves out-of-distribution gap. Another three-stage experiment preserves box-tightness best, achieving the highest mAP@0.5:0.95 among experiments in the study. The synthetic-data utility for safety-critical hand detection depends on the training procedure, and simple multi-stage experiments extract substantial real-deployment benefit from inpainted accessory data.