# The MECCANO Dataset: Understanding Human-Object Interactions from Egocentric Videos in an Industrial-like Domain

arXiv: https://arxiv.org/abs/2010.05654 · HTML: https://mlsanity.com/paper/2010.05654/

## Key facts

- Cited by 79 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Oct 2020.

## Built on by (most cited first, 30 of 79)

- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives](https://mlsanity.com/paper/2311.18259/) (2023)
- [Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activities](https://mlsanity.com/paper/2203.14712/) (2022)
- [H2O: Two Hands Manipulating Objects for First Person Interaction Recognition](https://mlsanity.com/paper/2104.11181/) (2021)
- [HoloAssist: an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real World](https://mlsanity.com/paper/2309.17024/) (2023)
- [EPIC-KITCHENS VISOR Benchmark: VIdeo Segmentations and Object Relations](https://mlsanity.com/paper/2209.13064/) (2022)
- [HD-EPIC: A Highly-Detailed Egocentric Video Dataset](https://mlsanity.com/paper/2502.04144/) (2025)
- [EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric View of Procedural Activities in Real World](https://mlsanity.com/paper/2403.16182/) (2024)
- [EgoLife: Towards Egocentric Life Assistant](https://mlsanity.com/paper/2503.03803/) (2025)
- [Temporal Action Segmentation: An Analysis of Modern Techniques](https://mlsanity.com/paper/2210.10352/) (2022)
- [An Outlook into the Future of Egocentric Vision](https://mlsanity.com/paper/2308.07123/) (2023)
- [My View is the Best View: Procedure Learning from Egocentric Videos](https://mlsanity.com/paper/2207.10883/) (2022)
- [CaptainCook4D: A Dataset for Understanding Errors in Procedural Activities](https://mlsanity.com/paper/2312.14556/) (2023)
- [IndustReal: A Dataset for Procedure Step Recognition Handling Execution Errors in Egocentric Videos in an Industrial-Like Setting](https://mlsanity.com/paper/2310.17323/) (2023)
- [EgoObjects: A Large-Scale Egocentric Dataset for Fine-Grained Object Understanding](https://mlsanity.com/paper/2309.08816/) (2023)
- [PREGO: online mistake detection in PRocedural EGOcentric videos](https://mlsanity.com/paper/2404.01933/) (2024)
- [Predicting the Future from First Person (Egocentric) Vision: A Survey](https://mlsanity.com/paper/2107.13411/) (2021)
- [MECCANO: A Multimodal Egocentric Dataset for Humans Behavior Understanding in the Industrial-like Domain](https://mlsanity.com/paper/2209.08691/) (2022)
- [EgoExo-Fitness: Towards Egocentric and Exocentric Full-Body Action Understanding](https://mlsanity.com/paper/2406.08877/) (2024)
- [Bongard-HOI: Benchmarking Few-Shot Visual Reasoning for Human-Object Interactions](https://mlsanity.com/paper/2205.13803/) (2022)
- [Egocentric Prediction of Action Target in 3D](https://mlsanity.com/paper/2203.13116/) (2022)
- [Visual Object Tracking in First Person Vision](https://mlsanity.com/paper/2209.13502/) (2022)
- [EgoExoBench: A Benchmark for First- and Third-person View Video Understanding in MLLMs](https://mlsanity.com/paper/2507.18342/) (2025)
- [Is First Person Vision Challenging for Object Tracking?](https://mlsanity.com/paper/2108.13665/) (2021)
- [EOC-Bench: Can MLLMs Identify, Recall, and Forecast Objects in an Egocentric World?](https://mlsanity.com/paper/2506.05287/) (2025)
- [Memory-efficient Streaming VideoLLMs for Real-time Procedural Video Understanding](https://mlsanity.com/paper/2504.13915/) (2025)
- [HA-ViD: A Human Assembly Video Dataset for Comprehensive Assembly Knowledge Understanding](https://mlsanity.com/paper/2307.05721/) (2023)
- [Every Mistake Counts in Assembly](https://mlsanity.com/paper/2307.16453/) (2023)
- [Leveraging Next-Active Objects for Context-Aware Anticipation in Egocentric Videos](https://mlsanity.com/paper/2308.08303/) (2023)
- [A Large-scale Study of Spatiotemporal Representation Learning with a New Benchmark on Action Recognition](https://mlsanity.com/paper/2303.13505/) (2023)

## Abstract (arXiv)

Wearable cameras allow to collect images and videos of humans interacting with the world. While human-object interactions have been thoroughly investigated in third person vision, the problem has been understudied in egocentric settings and in industrial scenarios. To fill this gap, we introduce MECCANO, the first dataset of egocentric videos to study human-object interactions in industrial-like settings. MECCANO has been acquired by 20 participants who were asked to build a motorbike model, for which they had to interact with tiny objects and tools. The dataset has been explicitly labeled for the task of recognizing human-object interactions from an egocentric perspective. Specifically, each interaction has been labeled both temporally (with action segments) and spatially (with active object bounding boxes). With the proposed dataset, we investigate four different tasks including 1) action recognition, 2) active object detection, 3) active object recognition and 4) egocentric human-object interaction detection, which is a revisited version of the standard human-object interaction detection task. Baseline results show that the MECCANO dataset is a challenging benchmark to study egocentric human-object interactions in industrial-like scenarios. We publicy release the dataset at https://iplab.dmi.unict.it/MECCANO.