# ESTANet: Efficient Online Error Detection in Procedural Videos via Prediction Inconsistency

arXiv: https://arxiv.org/abs/2606.25317 · HTML: https://mlsanity.com/paper/2606.25317/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 42 references that are in the map.
- First posted on arXiv 24 Jun 2026.

## Builds on (most cited first, 29 of 29)

- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentation](https://mlsanity.com/paper/1903.01945/) (2019)
- [Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activities](https://mlsanity.com/paper/2203.14712/) (2022)
- [ASFormer: Transformer for Action Segmentation](https://mlsanity.com/paper/2110.08568/) (2021)
- [HoloAssist: an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real World](https://mlsanity.com/paper/2309.17024/) (2023)
- [EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric View of Procedural Activities in Real World](https://mlsanity.com/paper/2403.16182/) (2024)
- [Long Short-Term Transformer for Online Action Detection](https://mlsanity.com/paper/2107.03377/) (2021)
- [OadTR: Online Action Detection with Transformers](https://mlsanity.com/paper/2106.11149/) (2021)
- [Temporal Recurrent Networks for Online Action Detection](https://mlsanity.com/paper/1811.07391/) (2018)
- [Real-time Online Video Detection with Temporal Smoothing Transformers](https://mlsanity.com/paper/2209.09236/) (2022)
- [Bridge-Prompt: Towards Ordinal Action Understanding in Instructional Videos](https://mlsanity.com/paper/2203.14104/) (2022)
- [Drop-DTW: Aligning Common Signal Between Sequences While Dropping Outliers](https://mlsanity.com/paper/2108.11996/) (2021)
- [Memory-and-Anticipation Transformer for Online Action Understanding](https://mlsanity.com/paper/2308.07893/) (2023)
- [PREGO: online mistake detection in PRocedural EGOcentric videos](https://mlsanity.com/paper/2404.01933/) (2024)
- [Learning to Discriminate Information for Online Action Detection](https://mlsanity.com/paper/1912.04461/) (2019)
- [StepFormer: Self-supervised Step Discovery and Localization in Instructional Videos](https://mlsanity.com/paper/2304.13265/) (2023)
- [SnAG: Scalable and Accurate Video Grounding](https://mlsanity.com/paper/2404.02257/) (2024)
- [Video-Mined Task Graphs for Keystep Recognition in Instructional Videos](https://mlsanity.com/paper/2307.08763/) (2023)
- [Fast Weakly Supervised Action Segmentation Using Mutual Consistency](https://mlsanity.com/paper/1904.03116/) (2019)
- [Differentiable Task Graph Learning: Procedural Activity Representation and Online Mistake Detection from Egocentric Videos](https://mlsanity.com/paper/2406.01486/) (2024)
- [Meta Reinforcement Learning with Autonomous Inference of Subtask Dependencies](https://mlsanity.com/paper/2001.00248/) (2020)
- [Every Mistake Counts in Assembly](https://mlsanity.com/paper/2307.16453/) (2023)
- [Multimodal Subtask Graph Generation from Instructional Videos](https://mlsanity.com/paper/2302.08672/) (2023)
- [Context-Enhanced Memory-Refined Transformer for Online Action Detection](https://mlsanity.com/paper/2503.18359/) (2025)
- [Modeling Multiple Normal Action Representations for Error Detection in Procedural Tasks](https://mlsanity.com/paper/2503.22405/) (2025)
- [Multi-Sentence Grounding for Long-term Instructional Video](https://mlsanity.com/paper/2312.14055/) (2023)
- [DeCafNet: Delegate and Conquer for Efficient Temporal Grounding in Long Videos](https://mlsanity.com/paper/2505.16376/) (2025)

## Built on by (most cited first, 1 of 1)

- [Hand-Aware Transition Modeling for Bimanual Procedural Anomaly Detection](https://mlsanity.com/paper/2609.21207/) (2026)

## Abstract (arXiv)

An efficient and accurate system for detecting errors in procedural tasks is crucial for supporting human needs in daily life, as it can provide instant notifications and guide people to correct mistakes. In this work, we study real-time online error detection in procedural videos from a simple but overlooked perspective: the prediction behavior of action detectors themselves. Instead of designing complex architectures or specialized supervision, we observe that action detectors naturally exhibit different prediction characteristics depending on their sensitivity to input dynamics and temporal context. We therefore propose ESTANet (Error-Sensitive and Temporally-vArying Network), a lightweight framework that detects errors by exploiting inconsistencies among action predictions produced by a small set of action detectors. We construct standard and error-sensitive action detectors that behave similarly on correct executions but respond differently when errors occur. Meanwhile, detectors operating with different temporal contexts further amplify prediction inconsistencies when the procedure deviates from the intended sequence. During inference, we detect errors by aggregating mismatches between standard and error-sensitive predictions through majority voting to flag frames that contain errors. Extensive experiments on EgoPER, Assembly-101-O, and EPIC-Tent-O demonstrate that ESTANet achieves state-of-the-art performance in online error detection while maintaining real-time efficiency with a lightweight architecture. Our results highlight that leveraging the intrinsic properties of action detectors can yield a powerful and practical solution for online error detection without increasing architectural design complexity.