# IMPACT-HOI: Supervisory Control for Onset-Anchored Partial HOI Event Construction

arXiv: https://arxiv.org/abs/2605.01666 · HTML: https://mlsanity.com/paper/2605.01666/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 38 references that are in the map.
- First posted on arXiv 3 May 2026.

## Builds on (most cited first, 30 of 34)

- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [Rescaling Egocentric Vision](https://mlsanity.com/paper/2006.13256/) (2020)
- [HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interaction](https://mlsanity.com/paper/2203.01577/) (2022)
- [ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation](https://mlsanity.com/paper/2204.13662/) (2022)
- [Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activities](https://mlsanity.com/paper/2203.14712/) (2022)
- [HOTR: End-to-End Human-Object Interaction Detection with Transformers](https://mlsanity.com/paper/2104.13682/) (2021)
- [EgoMimic: Scaling Imitation Learning via Egocentric Video](https://mlsanity.com/paper/2410.24221/) (2024)
- [The VIA Annotation Software for Images, Audio and Video](https://mlsanity.com/paper/1904.10699/) (2019)
- [QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Information](https://mlsanity.com/paper/2103.05399/) (2021)
- [EPIC-KITCHENS VISOR Benchmark: VIdeo Segmentations and Object Relations](https://mlsanity.com/paper/2209.13064/) (2022)
- [GEN-VLKT: Simplify Association and Enhance Interaction Understanding for HOI Detection](https://mlsanity.com/paper/2203.13954/) (2022)
- [Anticipative Feature Fusion Transformer for Multi-Modal Action Anticipation](https://mlsanity.com/paper/2210.12649/) (2022)
- [ST-HOI: A Spatial-Temporal Baseline for Human-Object Interaction Detection in Videos](https://mlsanity.com/paper/2105.11731/) (2021)
- [Learning the What and How of Annotation in Video Object Segmentation](https://mlsanity.com/paper/2311.04414/) (2023)
- [EgoNight: Towards Egocentric Vision Understanding at Night with a Challenging Benchmark](https://mlsanity.com/paper/2510.06218/) (2025)
- [Navigating Open Set Scenarios for Skeleton-based Action Recognition](https://mlsanity.com/paper/2312.06330/) (2023)
- [Learning Asynchronous and Sparse Human-Object Interaction in Videos](https://mlsanity.com/paper/2103.02758/) (2021)
- [Referring Atomic Video Action Recognition](https://mlsanity.com/paper/2407.01872/) (2024)
- [LIGHTEN: Learning Interactions with Graph and Hierarchical TEmporal Networks for HOI in videos](https://mlsanity.com/paper/2012.09402/) (2020)
- [Video-based Human-Object Interaction Detection from Tubelet Tokens](https://mlsanity.com/paper/2206.01908/) (2022)
- [Human-Object Interaction Prediction in Videos through Gaze Following](https://mlsanity.com/paper/2306.03597/) (2023)
- [Spatio-Temporal Interaction Graph Parsing Networks for Human-Object Interaction Recognition](https://mlsanity.com/paper/2108.08633/) (2021)
- [EgoSound: Benchmarking Sound Understanding in Egocentric Videos](https://mlsanity.com/paper/2602.14122/) (2026)
- [Semi-supervised Active Learning for Video Action Detection](https://mlsanity.com/paper/2312.07169/) (2023)
- [Adapting the Segment Anything Model During Usage in Novel Situations](https://mlsanity.com/paper/2404.08421/) (2024)
- [Skeleton-Based Human Action Recognition with Noisy Labels](https://mlsanity.com/paper/2403.09975/) (2024)
- [MICA: Multi-Agent Industrial Coordination Assistant](https://mlsanity.com/paper/2509.15237/) (2025)
- [IMPACT: A Dataset for Multi-Granularity Human Procedural Action Understanding in Industrial Assembly](https://mlsanity.com/paper/2604.10409/) (2026)
- [Snap, Segment, Deploy: A Visual Data and Detection Pipeline for Wearable Industrial Assistants](https://mlsanity.com/paper/2507.21072/) (2025)
- [HopaDIFF: Holistic-Partial Aware Fourier Conditioned Diffusion for Referring Human Action Segmentation in Multi-Person Scenarios](https://mlsanity.com/paper/2506.09650/) (2025)

## Built on by (most cited first, 2 of 2)

- [PROVIA: Procedure State Tracking for Online Mistake Detection in Egocentric Videos](https://mlsanity.com/paper/2609.20638/) (2026)
- [Hand-Aware Transition Modeling for Bimanual Procedural Anomaly Detection](https://mlsanity.com/paper/2609.21207/) (2026)

## Abstract (arXiv)

We present IMPACT-HOI, a mixed-initiative framework for annotating egocentric procedural video by constructing structured event graphs for Human-Object Interactions (HOI), motivated by the need for high-quality structured supervision for learning robot manipulation from human demonstration. IMPACT-HOI frames this task as the incremental resolution of a partially specified, onset-anchored event state. A trust-calibrated controller selects among direct queries, human-confirmed suggestions, and conservative completions based on empirical annotator behavior and evidence quality. A risk-bounded execution protocol, utilizing atomic rollback, ensures that human-confirmed decisions are preserved against conflicting automated updates. A user study with 9 participants shows a 13.5% reduction in manual annotation actions, a 46.67% event match rate, and zero confirmed-field violations under the studied protocol. The code will be made publicly available at https://github.com/541741106/IMPACT_HOI.