# IMPACT: A Dataset for Multi-Granularity Human Procedural Action Understanding in Industrial Assembly

arXiv: https://arxiv.org/abs/2604.10409 · HTML: https://mlsanity.com/paper/2604.10409/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 44 references that are in the map.
- First posted on arXiv 12 Apr 2026.

## Builds on (most cited first, 25 of 25)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [Scaling Egocentric Vision: The EPIC-KITCHENS Dataset](https://mlsanity.com/paper/1804.02748/) (2018)
- [VideoMAE V2: Scaling Video Masked Autoencoders with Dual Masking](https://mlsanity.com/paper/2303.16727/) (2023)
- [Temporal Convolutional Networks for Action Segmentation and Detection](https://mlsanity.com/paper/1611.05267/) (2016)
- [MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentation](https://mlsanity.com/paper/1903.01945/) (2019)
- [Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activities](https://mlsanity.com/paper/2203.14712/) (2022)
- [Anticipative Video Transformer](https://mlsanity.com/paper/2106.02036/) (2021)
- [HoloAssist: an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real World](https://mlsanity.com/paper/2309.17024/) (2023)
- [Ego-Exo: Transferring Visual Representations from Third-person to First-person Videos](https://mlsanity.com/paper/2104.07905/) (2021)
- [The IKEA ASM Dataset: Understanding People Assembling Furniture through Actions, Objects and Pose](https://mlsanity.com/paper/2007.00394/) (2020)
- [CaptainCook4D: A Dataset for Understanding Errors in Procedural Activities](https://mlsanity.com/paper/2312.14556/) (2023)
- [IndustReal: A Dataset for Procedure Step Recognition Handling Execution Errors in Egocentric Videos in an Industrial-Like Setting](https://mlsanity.com/paper/2310.17323/) (2023)
- [Learning Fine-grained View-Invariant Representations from Unpaired Ego-Exo Videos via Temporal Alignment](https://mlsanity.com/paper/2306.05526/) (2023)
- [MECCANO: A Multimodal Egocentric Dataset for Humans Behavior Understanding in the Industrial-like Domain](https://mlsanity.com/paper/2209.08691/) (2022)
- [Learning Object State Changes in Videos: An Open-World Perspective](https://mlsanity.com/paper/2312.11782/) (2023)
- [PALM: Predicting Actions through Language Models](https://mlsanity.com/paper/2311.17944/) (2023)
- [HA-ViD: A Human Assembly Video Dataset for Comprehensive Assembly Knowledge Understanding](https://mlsanity.com/paper/2307.05721/) (2023)
- [Synchronization is All You Need: Exocentric-to-Egocentric Transfer for Temporal Action Segmentation with Unlabeled Synchronized Video Pairs](https://mlsanity.com/paper/2312.02638/) (2023)
- [IndEgo: A Dataset of Industrial Scenarios and Collaborative Work for Egocentric Assistants](https://mlsanity.com/paper/2511.19684/) (2025)
- [MICA: Multi-Agent Industrial Coordination Assistant](https://mlsanity.com/paper/2509.15237/) (2025)
- [Bootstrap Your Own Views: Masked Ego-Exo Modeling for Fine-grained View-invariant Video Representations](https://mlsanity.com/paper/2503.19706/) (2025)
- [Snap, Segment, Deploy: A Visual Data and Detection Pipeline for Wearable Industrial Assistants](https://mlsanity.com/paper/2507.21072/) (2025)
- [Learning to Recognize Correctly Completed Procedure Steps in Egocentric Assembly Videos through Spatio-Temporal Modeling](https://mlsanity.com/paper/2510.12385/) (2025)

## Built on by (most cited first, 8 of 8)

- [IMPACT-HOI: Supervisory Control for Onset-Anchored Partial HOI Event Construction](https://mlsanity.com/paper/2605.01666/) (2026)
- [PROVIA: Procedure State Tracking for Online Mistake Detection in Egocentric Videos](https://mlsanity.com/paper/2609.20638/) (2026)
- [Reachability Is Not Generalization: Understanding Verb--Noun Decomposition in Assembly Action Recognition](https://mlsanity.com/paper/2610.00064/) (2026)
- [IMPACT-Scribe: Interactive Temporal Action Segmentation with Boundary Scribbles and Query Planning](https://mlsanity.com/paper/2605.01668/) (2026)
- [INSPECT: Learning Robot View Selection from Assistant Use](https://mlsanity.com/paper/2609.20615/) (2026)
- [Hand-Aware Transition Modeling for Bimanual Procedural Anomaly Detection](https://mlsanity.com/paper/2609.21207/) (2026)
- [Algebraic Consistency Alone Does Not Certify Temporal Structure in Latent Action Models](https://mlsanity.com/paper/2609.23478/) (2026)
- [HEIR: Learning Human-Entity Interactions with Functional Roles](https://mlsanity.com/paper/2609.35955/) (2026)

## Abstract (arXiv)

We introduce IMPACT, a synchronized five-view RGB-D dataset for deployment-oriented industrial procedural understanding, built around real assembly and disassembly of a commercial angle grinder with professional-grade tools. To our knowledge, IMPACT is the first real industrial assembly benchmark that jointly provides synchronized ego-exo RGB-D capture, decoupled bimanual annotation, compliance-aware state tracking, and explicit anomaly--recovery supervision within a single real industrial workflow. It comprises 112 trials from 13 participants totaling 39.5 hours, with multi-route execution governed by a partial-order prerequisite graph, a six-category anomaly taxonomy, and operator cognitive load measured via NASA-TLX. The annotation hierarchy links hand-specific atomic actions to coarse procedural steps, component assembly states, and per-hand compliance phases, with synchronized null spans across views to decouple perceptual limitations from algorithmic failure. Systematic baselines reveal fundamental limitations that remain invisible to single-task benchmarks, particularly under realistic deployment conditions that involve incomplete observations, flexible execution paths, and corrective behavior. The full dataset, annotations, and evaluation code are available at https://github.com/Kratos-Wen/IMPACT.