# Egocentric Human-Object Interaction Detection Exploiting Synthetic Data

arXiv: https://arxiv.org/abs/2204.07061 · HTML: https://mlsanity.com/paper/2204.07061/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 35 references that are in the map.
- First posted on arXiv 14 Apr 2022.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [Scaling Egocentric Vision: The EPIC-KITCHENS Dataset](https://mlsanity.com/paper/1804.02748/) (2018)
- [Rescaling Egocentric Vision](https://mlsanity.com/paper/2006.13256/) (2020)
- [Learning joint reconstruction of hands and manipulated objects](https://mlsanity.com/paper/1904.05767/) (2019)
- [Understanding Human Hands in Contact at Internet Scale](https://mlsanity.com/paper/2006.06669/) (2020)
- [Learning to Detect Human-Object Interactions](https://mlsanity.com/paper/1702.05448/) (2017)
- [First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotations](https://mlsanity.com/paper/1704.02463/) (2017)
- [Detecting and Recognizing Human-Object Interactions](https://mlsanity.com/paper/1704.07333/) (2017)
- [Visual Semantic Role Labeling](https://mlsanity.com/paper/1505.04474/) (2015)
- [GANerated Hands for Real-time 3D Hand Tracking from Monocular RGB](https://mlsanity.com/paper/1712.01057/) (2017)
- [PPDM: Parallel Point Detection and Matching for Real-time Human-Object Interaction Detection](https://mlsanity.com/paper/1912.12898/) (2019)
- [Rolling-Unrolling LSTMs for Action Anticipation from First-Person Video](https://mlsanity.com/paper/2005.02190/) (2020)
- [Efficient Two-Stage Detection of Human-Object Interactions with a Novel Unary-Pairwise Transformer](https://mlsanity.com/paper/2112.01838/) (2021)
- [Real-time Hand Tracking under Occlusion from an Egocentric RGB-D Sensor](https://mlsanity.com/paper/1704.02201/) (2017)
- [The MECCANO Dataset: Understanding Human-Object Interactions from Egocentric Videos in an Industrial-like Domain](https://mlsanity.com/paper/2010.05654/) (2020)
- [Holistic++ Scene Understanding: Single-view 3D Holistic Scene Parsing and Human Pose Estimation with Human-Object Interaction and Physical Commonsense](https://mlsanity.com/paper/1909.01507/) (2019)
- [In the Eye of the Beholder: Gaze and Actions in First Person Video](https://mlsanity.com/paper/2006.00626/) (2020)
- [Next-Active-Object prediction from Egocentric Videos](https://mlsanity.com/paper/1904.05250/) (2019)
- [The Evolution of First Person Vision Methods: A Survey](https://mlsanity.com/paper/1409.1484/) (2014)
- [Improved Handling of Motion Blur in Online Object Detection](https://mlsanity.com/paper/2011.14448/) (2020)
- [Sequential Voting with Relational Box Fields for Active Object Detection](https://mlsanity.com/paper/2110.11524/) (2021)
- [Understanding Egocentric Hand-Object Interactions from Hand Pose Estimation](https://mlsanity.com/paper/2109.14657/) (2021)
- [Egocentric Visitors Localization in Cultural Sites](https://mlsanity.com/paper/1904.05264/) (2019)
- [The Object at Hand: Automated Editing for Mixed Reality Video Guidance from Hand-Object Interactions](https://mlsanity.com/paper/2109.14744/) (2021)

## Built on by (most cited first, 11 of 11)

- [An Outlook into the Future of Egocentric Vision](https://mlsanity.com/paper/2308.07123/) (2023)
- [EgoObjects: A Large-Scale Egocentric Dataset for Fine-Grained Object Understanding](https://mlsanity.com/paper/2309.08816/) (2023)
- [PromptonomyViT: Multi-Task Prompt Learning Improves Video Transformers using Synthetic Scene Data](https://mlsanity.com/paper/2212.04821/) (2022)
- [Exploiting Multimodal Synthetic Data for Egocentric Human-Object Interaction Detection in an Industrial Scenario](https://mlsanity.com/paper/2306.12152/) (2023)
- [ENIGMA-51: Towards a Fine-Grained Understanding of Human-Object Interactions in Industrial Scenarios](https://mlsanity.com/paper/2309.14809/) (2023)
- [Are Synthetic Data Useful for Egocentric Hand-Object Interaction Detection?](https://mlsanity.com/paper/2312.02672/) (2023)
- [ANNEXE: Unified Analyzing, Answering, and Pixel Grounding for Egocentric Interaction](https://mlsanity.com/paper/2504.01472/) (2025)
- [EgoInteract: Synthetic Egocentric Videos Generation for Interaction Understanding and Anticipation](https://mlsanity.com/paper/2605.18214/) (2026)
- [Learning Egocentric In-Hand Object Segmentation through Weak Supervision from Human Narrations](https://mlsanity.com/paper/2509.26004/) (2025)
- [GlovEgo-HOI: Bridging the Synthetic-to-Real Gap for Industrial Egocentric Human-Object Interaction Detection](https://mlsanity.com/paper/2601.09528/) (2026)
- [Leveraging Synthetic Data for Enhancing Egocentric Hand-Object Interaction Detection](https://mlsanity.com/paper/2603.29733/) (2026)

## Abstract (arXiv)

We consider the problem of detecting Egocentric HumanObject Interactions (EHOIs) in industrial contexts. Since collecting and labeling large amounts of real images is challenging, we propose a pipeline and a tool to generate photo-realistic synthetic First Person Vision (FPV) images automatically labeled for EHOI detection in a specific industrial scenario. To tackle the problem of EHOI detection, we propose a method that detects the hands, the objects in the scene, and determines which objects are currently involved in an interaction. We compare the performance of our method with a set of state-of-the-art baselines. Results show that using a synthetic dataset improves the performance of an EHOI detection system, especially when few real data are available. To encourage research on this topic, we publicly release the proposed dataset at the following url: https://iplab.dmi.unict.it/EHOI_SYNTH/.