mlsanityData to 7 Oct 2026

ENIGMA-51: Towards a Fine-Grained Understanding of Human-Object Interactions in Industrial Scenarios

arXiv 2309.14809 ↗First posted 26 Sep 2023cs.CVcited by 11

Not on the map. Why?

Abstract (arXiv)

ENIGMA-51 is a new egocentric dataset acquired in an industrial scenario by 19 subjects who followed instructions to complete the repair of electrical boards using industrial tools (e.g., electric screwdriver) and equipments (e.g., oscilloscope). The 51 egocentric video sequences are densely annotated with a rich set of labels that enable the systematic study of human behavior in the industrial domain. We provide benchmarks on four tasks related to human behavior: 1) untrimmed temporal detection of human-object interactions, 2) egocentric human-object interaction detection, 3) short-term object interaction anticipation and 4) natural language understanding of intents and entities. Baseline results show that the ENIGMA-51 dataset poses a challenging benchmark to study human behavior in industrial scenarios. We publicly release the dataset at https://iplab.dmi.unict.it/ENIGMA-51.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 48 of its 67 references that are in the map. First posted on arXiv 26 Sep 2023.

Builds on

48 of 67 references.

References · 48

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  5. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  6. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  7. 2018SlowFast Networks for Video Recognitioncited by 1,862
  8. 2020YOLOv4: Optimal Speed and Accuracy of Object Detectioncited by 1,624
  9. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,306
  10. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,305
  11. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 789
  12. 2020X3D: Expanding Architectures for Efficient Video Recognitioncited by 635
36 more
  1. 2020Rescaling Egocentric Visioncited by 529
  2. 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 516
  3. 2022InternVideo: General Video Foundation Models via Generative and Discriminative Learningcited by 441
  4. 2023Segment Anything in High Qualitycited by 373
  5. 2022HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interactioncited by 321
  6. 2022ActionFormer: Localizing Moments of Actions with Transformerscited by 308
  7. 2022ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulationcited by 306
  8. 2022Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activitiescited by 284
  9. 2020Understanding Human Hands in Contact at Internet Scalecited by 281
  10. 2017Temporal Segment Networks for Action Recognition in Videoscited by 279
  11. 2019Cross-task weakly supervised learning from instructional videoscited by 199
  12. 2019BERT for Joint Intent Classification and Slot Fillingcited by 195
  13. 2023TriDet: Temporal Action Detection with Relative Boundary Modelingcited by 141
  14. 2022EPIC-KITCHENS VISOR Benchmark: VIdeo Segmentations and Object Relationscited by 139
  15. 2019A Novel Bi-directional Interrelated Model for Joint Intent Detection and Slot Fillingcited by 85
  16. 2022My View is the Best View: Procedure Learning from Egocentric Videoscited by 81
  17. 2020The MECCANO Dataset: Understanding Human-Object Interactions from Egocentric Videos in an Industrial-like Domaincited by 79
  18. 2022Fine-Grained Egocentric Hand-Object Segmentation: Dataset, Model, and Applicationscited by 78
  19. 2017Rasa: Open Source Language Understanding and Dialogue Managementcited by 67
  20. 2023Efficient Sequence Transduction by Jointly Predicting Tokens and Durationscited by 56
  21. 2019Next-Active-Object prediction from Egocentric Videoscited by 55
  22. 2022InternVideo-Ego4D: A Pack of Champion Solutions to Ego4D Challengescited by 48
  23. 2022Breaking the "Object" in Video Object Segmentationcited by 48
  24. 2021Forecasting Action through Contact Representations from First Person Videocited by 46
  25. 2022MECCANO: A Multimodal Egocentric Dataset for Humans Behavior Understanding in the Industrial-like Domaincited by 36
  26. 2020DIET: Lightweight Language Understanding for Dialogue Systemscited by 35
  27. 2023StillFast: An End-to-End Approach for Short-Term Object Interaction Anticipationcited by 23
  28. 2019StartNet: Online Detection of Action Start in Untrimmed Videoscited by 21
  29. 2023Summarize the Past to Predict the Future: Natural Language Descriptions of Context Boost Multimodal Object Interaction Anticipationcited by 21
  30. 2020WOAD: Weakly Supervised Online Action Detection in Untrimmed Videoscited by 19
  31. 2018Online Detection of Action Start in Untrimmed, Streaming Videoscited by 17
  32. 2020Egocentric Object Manipulation Graphscited by 17
  33. 2023Exploiting Multimodal Synthetic Data for Egocentric Human-Object Interaction Detection in an Industrial Scenariocited by 14
  34. 2019Hierarchical Multi-Task Natural Language Understanding for Cross-domain Conversational AI: HERMIT NLUcited by 13
  35. 2022Egocentric Human-Object Interaction Detection Exploiting Synthetic Datacited by 11
  36. 2021Egocentric Hand-object Interaction Detection and Applicationcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 4; 2025: 3; 2026: 3.

11 papers.

  1. 2024HOT3D: Hand and Object Tracking in 3D from Egocentric Multi-View Videoscited by 122
  2. 2025EgoLife: Towards Egocentric Life Assistantcited by 86
  3. 2024PREGO: online mistake detection in PRocedural EGOcentric videoscited by 36
  4. 2024Multimodal Cross-Domain Few-Shot Learning for Egocentric Action Recognitioncited by 14
  5. 2023Are Synthetic Data Useful for Egocentric Hand-Object Interaction Detection?cited by 10
  6. 2024TI-PREGO: Chain of Thought and In-Context Learning for Online Mistake Detection in PRocedural EGOcentric Videoscited by 10
  7. 2026Data Pyramid for Embodied Manipulation: A Surveycited by 4
  8. 2025Ego-EXTRA: video-language Egocentric Dataset for EXpert-TRAinee assistancecited by 3
  9. 2026ENIGMA-360: An Ego-Exo Dataset for Human Behavior Understanding in Industrial Scenarioscited by 2
  10. 2026Leveraging Gaze and Set-of-Mark in VLLMs for Human-Object Interaction Anticipation from Egocentric Videoscited by 2
  11. 2025A Real-Time System for Egocentric Hand-Object Interaction Detection in Industrial Domainscited by 0