mlsanityData to 14 Oct 2026

MECCANO: A Multimodal Egocentric Dataset for Humans Behavior Understanding in the Industrial-like Domain

arXiv 2209.08691 ↗First posted 19 Sep 2022cs.CVcited by 36

Not on the map. Why?

Abstract (arXiv)

Wearable cameras allow to acquire images and videos from the user's perspective. These data can be processed to understand humans behavior. Despite human behavior analysis has been thoroughly investigated in third person vision, it is still understudied in egocentric settings and in particular in industrial scenarios. To encourage research in this field, we present MECCANO, a multimodal dataset of egocentric videos to study humans behavior understanding in industrial-like settings. The multimodality is characterized by the presence of gaze signals, depth maps and RGB videos acquired simultaneously with a custom headset. The dataset has been explicitly labeled for fundamental tasks in the context of human behavior understanding from a first person view, such as recognizing and anticipating human-object interactions. With the MECCANO dataset, we explored five different tasks including 1) Action Recognition, 2) Active Objects Detection and Recognition, 3) Egocentric Human-Objects Interaction Detection, 4) Action Anticipation and 5) Next-Active Objects Detection. We propose a benchmark aimed to study human behavior in the considered industrial-like scenario which demonstrates that the investigated tasks and the considered scenario are challenging for state-of-the-art algorithms. To support research in this field, we publicy release the dataset at https://iplab.dmi.unict.it/MECCANO/.

Key facts. Cited by 36 papers in the map (data through 14 Oct 2026). Builds on 58 of its 107 references that are in the map. First posted on arXiv 19 Sep 2022.

Builds on

58 of 107 references.

References · 58

  1. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  2. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  3. 2015Fast R-CNNcited by 2,933
  4. 2018YOLOv3: An Incremental Improvementcited by 2,659
  5. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,162
  6. 2017The Kinetics Human Action Video Datasetcited by 1,909
  7. 2018SlowFast Networks for Video Recognitioncited by 1,867
  8. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  9. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,143
  10. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,103
  11. 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,068
  12. 2016Temporal Segment Networks: Towards Good Practices for Deep Action Recognitioncited by 902
38 more
  1. 2018TSM: Temporal Shift Module for Efficient Video Understandingcited by 856
  2. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 795
  3. 2020X3D: Expanding Architectures for Efficient Video Recognitioncited by 637
  4. 2019NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understandingcited by 533
  5. 2017Learning Spatio-Temporal Representation with Pseudo-3D Residual Networkscited by 387
  6. 2016Convolutional Two-Stream Network Fusion for Video Action Recognitioncited by 384
  7. 2017Temporal Relational Reasoning in Videoscited by 324
  8. 2019A Short Note on the Kinetics-700 Human Action Datasetcited by 310
  9. 2022Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activitiescited by 287
  10. 2020Understanding Human Hands in Contact at Internet Scalecited by 281
  11. 2021H2O: Two Hands Manipulating Objects for First Person Interaction Recognitioncited by 254
  12. 2017Learning to Detect Human-Object Interactionscited by 240
  13. 2017First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotationscited by 230
  14. 2019EPIC-Fusion: Audio-Visual Temporal Binding for Egocentric Action Recognitioncited by 226
  15. 2020The EPIC-KITCHENS Dataset: Collection, Challenges and Baselinescited by 225
  16. 2021Anticipative Video Transformercited by 200
  17. 2017Detecting and Recognizing Human-Object Interactionscited by 199
  18. 2015Visual Semantic Role Labelingcited by 190
  19. 2018Learning Human-Object Interactions by Graph Parsing Neural Networkscited by 176
  20. 2019The VIA Annotation Software for Images, Audio and Videocited by 167
  21. 2019PPDM: Parallel Point Detection and Matching for Real-time Human-Object Interaction Detectioncited by 143
  22. 2018Grounded Human-Object Interaction Hotspots from Videocited by 129
  23. 2020Rolling-Unrolling LSTMs for Action Anticipation from First-Person Videocited by 120
  24. 2020EGO-TOPO: Environment Affordances from Egocentric Videocited by 111
  25. 2020Learning Human-Object Interaction Detection using Interaction Pointscited by 108
  26. 2019What Would You Expect? Anticipating Egocentric Actions with Rolling-Unrolling LSTMs and Modality Attentioncited by 105
  27. 2018Actor and Observer: Joint Modeling of First and Third-Person Videoscited by 103
  28. 2016Spatiotemporal Residual Networks for Video Action Recognitioncited by 91
  29. 2018Timeception for Complex Action Recognitioncited by 89
  30. 2016Long-term Temporal Convolutions for Action Recognitioncited by 84
  31. 2022My View is the Best View: Procedure Learning from Egocentric Videoscited by 81
  32. 2020The MECCANO Dataset: Understanding Human-Object Interactions from Egocentric Videos in an Industrial-like Domaincited by 79
  33. 2017RED: Reinforced Encoder-Decoder Networks for Action Anticipationcited by 78
  34. 2019Next-Active-Object prediction from Egocentric Videoscited by 55
  35. 2022In the Eye of Transformer: Global-Local Correlation for Egocentric Gaze Estimationcited by 51
  36. 2021Forecasting Action through Contact Representations from First Person Videocited by 46
  37. 2014Histogram of Oriented Principal Components for Cross-View Action Recognitioncited by 27
  38. 2019Gaze-in-wild: A dataset for studying eye and head coordination in everyday activitiescited by 25

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 8; 2024: 7; 2025: 7; 2026: 13.

36 papers.

Sort:

By area:

  1. 2022EPIC-KITCHENS VISOR Benchmark: VIdeo Segmentations and Object Relationscited by 140
  2. 2023An Outlook into the Future of Egocentric Visioncited by 84
  3. 2023IndustReal: A Dataset for Procedure Step Recognition Handling Execution Errors in Egocentric Videos in an Industrial-Like Settingcited by 48
  4. 2023Is ImageNet worth 1 video? Learning strong image encoders from 1 long unlabelled videocited by 40
  5. 2023StillFast: An End-to-End Approach for Short-Term Object Interaction Anticipationcited by 23
  6. 2023Exploiting Multimodal Synthetic Data for Egocentric Human-Object Interaction Detection in an Industrial Scenariocited by 14
  7. 2024Multimodal Cross-Domain Few-Shot Learning for Egocentric Action Recognitioncited by 14
  8. 2023ENIGMA-51: Towards a Fine-Grained Understanding of Human-Object Interactions in Industrial Scenarioscited by 11
  9. 2023A Survey on Deep Learning Techniques for Action Anticipationcited by 11
  10. 2024HOI-Ref: Hand-Object Interaction Referral in Egocentric Visioncited by 9
  11. 2024Active Object Detection with Knowledge Aggregation and Distillation from Large Modelscited by 8
  12. 2026IMPACT: A Dataset for Multi-Granularity Human Procedural Action Understanding in Industrial Assemblycited by 8
24 more
  1. 2024Uncertainty-boosted Robust Video Activity Anticipationcited by 7
  2. 2025HiERO: understanding the hierarchy of human behavior enhances reasoning on egocentric videoscited by 7
  3. 2024Are you Struggling? Dataset and Baselines for Struggle Determination in Assembly Videoscited by 5
  4. 2024EgoOops: A Dataset for Mistake Action Detection from Egocentric Videos referring to Procedural Textscited by 5
  5. 2025Action Anticipation at a Glimpse: To What Extent Can Multimodal Cues Replace Video?cited by 4
  6. 2025Ego-EXTRA: video-language Egocentric Dataset for EXpert-TRAinee assistancecited by 4
  7. 2025Egocentric Human-Object Interaction Detection: A New Benchmark and Methodcited by 3
  8. 2026Understanding Multimodal Complementarity for Single-Frame Action Anticipationcited by 2
  9. 2026ENIGMA-360: An Ego-Exo Dataset for Human Behavior Understanding in Industrial Scenarioscited by 2
  10. 2026Leveraging Gaze and Set-of-Mark in VLLMs for Human-Object Interaction Anticipation from Egocentric Videoscited by 2
  11. 2023Egocentric RGB+Depth Action Recognition in Industry-Like Settingscited by 1
  12. 2024ADL4D: Towards A Contextually Rich Dataset for 4D Activities of Daily Livingcited by 1
  13. 2026EgoInteract: Synthetic Egocentric Videos Generation for Interaction Understanding and Anticipationcited by 1
  14. 2026Reachability Is Not Generalization: Understanding Verb--Noun Decomposition in Assembly Action Recognitioncited by 1
  15. 2025A Multimodal Dataset for Enhancing Industrial Task Monitoring and Engagement Predictioncited by 0
  16. 2025Learning Egocentric In-Hand Object Segmentation through Weak Supervision from Human Narrationscited by 0
  17. 2025Vision-Based Mistake Analysis in Procedural Activities: A Review of Advances and Challengescited by 0
  18. 2026GlovEgo-HOI: Bridging the Synthetic-to-Real Gap for Industrial Egocentric Human-Object Interaction Detectioncited by 0
  19. 2026Show, Don't Tell: Detecting Novel Objects by Watching Human Videoscited by 0
  20. 2026From Instructions to Assistance: a Dataset Aligning Instruction Manuals with Assembly Videos for Evaluating Multimodal LLMscited by 0
  21. 2026Divide, Deliberate, Decide: A Multi-Agent Framework for Fine-Grained Egocentric Action Recognitioncited by 0
  22. 2026AnyGroundBench: A Specialized-Domain Benchmark for Video Grounding in Vision-Language Modelscited by 0
  23. 2026Event ActivityNet: A Large-Scale Simulated-Event Benchmark for Untrimmed Action Understandingcited by 0
  24. 2026EgoTools: Towards Tool-Centric Reasoning in Real-World Egocentric Videoscited by 0