# Virtual to Real adaptation of Pedestrian Detectors

arXiv: https://arxiv.org/abs/2001.03032 · HTML: https://mlsanity.com/paper/2001.03032/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jan 2020.

## Built on by (most cited first, 9 of 9)

- [Combining EfficientNet and Vision Transformers for Video Deepfake Detection](https://mlsanity.com/paper/2107.02612/) (2021)
- [MOTSynth: How Can Synthetic Data Help Pedestrian Detection and Tracking?](https://mlsanity.com/paper/2108.09518/) (2021)
- [Detection Recovery in Online Multi-Object Tracking with Sparse Graph Tracker](https://mlsanity.com/paper/2205.00968/) (2022)
- [Recurrent Vision Transformer for Solving Visual Reasoning Problems](https://mlsanity.com/paper/2111.14576/) (2021)
- [MOBDrone: a Drone Video Dataset for Man OverBoard Rescue](https://mlsanity.com/paper/2203.07973/) (2022)
- [A Spatio-Temporal Attentive Network for Video-Based Crowd Counting](https://mlsanity.com/paper/2208.11339/) (2022)
- [CrowdSim2: an Open Synthetic Benchmark for Object Detectors](https://mlsanity.com/paper/2304.05090/) (2023)
- [Development of a Realistic Crowd Simulation Environment for Fine-grained Validation of People Tracking Methods](https://mlsanity.com/paper/2304.13403/) (2023)
- [ParGANDA: Making Synthetic Pedestrians A Reality For Object Detection](https://mlsanity.com/paper/2307.11360/) (2023)

## Abstract (arXiv)

Pedestrian detection through Computer Vision is a building block for a multitude of applications. Recently, there was an increasing interest in Convolutional Neural Network-based architectures for the execution of such a task. One of these supervised networks' critical goals is to generalize the knowledge learned during the training phase to new scenarios with different characteristics. A suitably labeled dataset is essential to achieve this purpose. The main problem is that manually annotating a dataset usually requires a lot of human effort, and it is costly. To this end, we introduce ViPeD (Virtual Pedestrian Dataset), a new synthetically generated set of images collected with the highly photo-realistic graphical engine of the video game GTA V - Grand Theft Auto V, where annotations are automatically acquired. However, when training solely on the synthetic dataset, the model experiences a Synthetic2Real Domain Shift leading to a performance drop when applied to real-world images. To mitigate this gap, we propose two different Domain Adaptation techniques suitable for the pedestrian detection task, but possibly applicable to general object detection. Experiments show that the network trained with ViPeD can generalize over unseen real-world scenarios better than the detector trained over real-world data, exploiting the variety of our synthetic dataset. Furthermore, we demonstrate that with our Domain Adaptation techniques, we can reduce the Synthetic2Real Domain Shift, making closer the two domains and obtaining a performance improvement when testing the network over the real-world images. The code, the models, and the dataset are made freely available at https://ciampluca.github.io/viped/