IMPACT: A Dataset for Multi-Granularity Human Procedural Action Understanding in Industrial Assembly
Not on the map. Why?
Abstract (arXiv)
We introduce IMPACT, a synchronized five-view RGB-D dataset for deployment-oriented industrial procedural understanding, built around real assembly and disassembly of a commercial angle grinder with professional-grade tools. To our knowledge, IMPACT is the first real industrial assembly benchmark that jointly provides synchronized ego-exo RGB-D capture, decoupled bimanual annotation, compliance-aware state tracking, and explicit anomaly--recovery supervision within a single real industrial workflow. It comprises 112 trials from 13 participants totaling 39.5 hours, with multi-route execution governed by a partial-order prerequisite graph, a six-category anomaly taxonomy, and operator cognitive load measured via NASA-TLX. The annotation hierarchy links hand-specific atomic actions to coarse procedural steps, component assembly states, and per-hand compliance phases, with synchronized null spans across views to decouple perceptual limitations from algorithmic failure. Systematic baselines reveal fundamental limitations that remain invisible to single-task benchmarks, particularly under realistic deployment conditions that involve incomplete observations, flexible execution paths, and corrective behavior. The full dataset, annotations, and evaluation code are available at https://github.com/Kratos-Wen/IMPACT.
Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 25 of its 44 references that are in the map. First posted on arXiv 12 Apr 2026.
Builds on
25 of 44 references.
References · 25
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
- 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
- 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
- 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 789
- 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 516
- 2016Temporal Convolutional Networks for Action Segmentation and Detectioncited by 504
- 2019MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentationcited by 370
- 2022Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activitiescited by 284
- 2021Anticipative Video Transformercited by 197
- 2023HoloAssist: an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real Worldcited by 159
- 2021Ego-Exo: Transferring Visual Representations from Third-person to First-person Videoscited by 98
- 2020The IKEA ASM Dataset: Understanding People Assembling Furniture through Actions, Objects and Posecited by 95
13 more
- 2023CaptainCook4D: A Dataset for Understanding Errors in Procedural Activitiescited by 47
- 2023Learning Fine-grained View-Invariant Representations from Unpaired Ego-Exo Videos via Temporal Alignmentcited by 46
- 2023IndustReal: A Dataset for Procedure Step Recognition Handling Execution Errors in Egocentric Videos in an Industrial-Like Settingcited by 46
- 2022MECCANO: A Multimodal Egocentric Dataset for Humans Behavior Understanding in the Industrial-like Domaincited by 36
- 2023Learning Object State Changes in Videos: An Open-World Perspectivecited by 36
- 2023PALM: Predicting Actions through Language Modelscited by 28
- 2023HA-ViD: A Human Assembly Video Dataset for Comprehensive Assembly Knowledge Understandingcited by 18
- 2023Synchronization is All You Need: Exocentric-to-Egocentric Transfer for Temporal Action Segmentation with Unlabeled Synchronized Video Pairscited by 17
- 2025IndEgo: A Dataset of Industrial Scenarios and Collaborative Work for Egocentric Assistantscited by 9
- 2025MICA: Multi-Agent Industrial Coordination Assistantcited by 8
- 2025Bootstrap Your Own Views: Masked Ego-Exo Modeling for Fine-grained View-invariant Video Representationscited by 6
- 2025Snap, Segment, Deploy: A Visual Data and Detection Pipeline for Wearable Industrial Assistantscited by 6
- 2025Learning to Recognize Correctly Completed Procedure Steps in Egocentric Assembly Videos through Spatio-Temporal Modelingcited by 3
Built on by
Citing papers per year: 2026: 8.
8 papers.
- 2026IMPACT-HOI: Supervisory Control for Onset-Anchored Partial HOI Event Constructioncited by 2
- 2026PROVIA: Procedure State Tracking for Online Mistake Detection in Egocentric Videoscited by 1
- 2026IMPACT-Scribe: Interactive Temporal Action Segmentation with Boundary Scribbles and Query Planningcited by 0
- 2026INSPECT: Learning Robot View Selection from Assistant Usecited by 0
- 2026Hand-Aware Transition Modeling for Bimanual Procedural Anomaly Detectioncited by 0
- 2026Algebraic Consistency Alone Does Not Certify Temporal Structure in Latent Action Modelscited by 0
- 2026HEIR: Learning Human-Entity Interactions with Functional Rolescited by 0
- 2026Reachability Is Not Generalization: Understanding Verb--Noun Decomposition in Assembly Action Recognitioncited by 0
- 2026HEIR: Learning Human-Entity Interactions with Functional Roles28 Sep 2026
- 2026Algebraic Consistency Alone Does Not Certify Temporal Structure in Latent Action Models20 Sep 2026
- 2026Hand-Aware Transition Modeling for Bimanual Procedural Anomaly Detection18 Sep 2026
- 2026PROVIA: Procedure State Tracking for Online Mistake Detection in Egocentric Videos17 Sep 2026
- 2026INSPECT: Learning Robot View Selection from Assistant Use17 Sep 2026
- 2026Reachability Is Not Generalization: Understanding Verb--Noun Decomposition in Assembly Action Recognition4 Sep 2026
- 2026IMPACT-Scribe: Interactive Temporal Action Segmentation with Boundary Scribbles and Query Planning3 May 2026
- 2026IMPACT-HOI: Supervisory Control for Onset-Anchored Partial HOI Event Construction3 May 2026