# How to Correctly Make Mistakes: A Framework for Constructing and Benchmarking Mistake Aware Egocentric Procedural Videos

arXiv: https://arxiv.org/abs/2604.15134 · HTML: https://mlsanity.com/paper/2604.15134/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 50 references that are in the map.
- First posted on arXiv 16 Apr 2026.

## Builds on (most cited first, 17 of 17)

- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Models](https://mlsanity.com/paper/2402.17177/) (2024)
- [Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activities](https://mlsanity.com/paper/2203.14712/) (2022)
- [HoloAssist: an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real World](https://mlsanity.com/paper/2309.17024/) (2023)
- [AnyV2V: A Tuning-Free Framework For Any Video-to-Video Editing Tasks](https://mlsanity.com/paper/2403.14468/) (2024)
- [My View is the Best View: Procedure Learning from Egocentric Videos](https://mlsanity.com/paper/2207.10883/) (2022)
- [IndustReal: A Dataset for Procedure Step Recognition Handling Execution Errors in Egocentric Videos in an Industrial-Like Setting](https://mlsanity.com/paper/2310.17323/) (2023)
- [Learning Procedure-aware Video Representation from Instructional Videos and Their Narrations](https://mlsanity.com/paper/2303.17839/) (2023)
- [PREGO: online mistake detection in PRocedural EGOcentric videos](https://mlsanity.com/paper/2404.01933/) (2024)
- [Learning and Verification of Task Structure in Instructional Videos](https://mlsanity.com/paper/2303.13519/) (2023)
- [Every Mistake Counts in Assembly](https://mlsanity.com/paper/2307.16453/) (2023)
- [EasyV2V: A High-quality Instruction-based Video Editing Framework](https://mlsanity.com/paper/2512.16920/) (2025)
- [Procedural Mistake Detection via Action Effect Modeling](https://mlsanity.com/paper/2512.03474/) (2025)
- [EgoOops: A Dataset for Mistake Action Detection from Egocentric Videos referring to Procedural Texts](https://mlsanity.com/paper/2410.05343/) (2024)
- [Transparent and Coherent Procedural Mistake Detection](https://mlsanity.com/paper/2412.11927/) (2024)
- [Predicting Implicit Arguments in Procedural Video Instructions](https://mlsanity.com/paper/2505.21068/) (2025)

## Built on by (most cited first, 2 of 2)

- [Hand-Aware Transition Modeling for Bimanual Procedural Anomaly Detection](https://mlsanity.com/paper/2609.21207/) (2026)
- [LabInstruct: Benchmarking Situated Instructional Video Generation for Lab Procedures](https://mlsanity.com/paper/2610.14020/) (2026)

## Abstract (arXiv)

Reliable procedural monitoring in video requires exposure to naturally occurring human errors and the recoveries that follow. In egocentric recordings, mistakes are often partially occluded by hands and revealed through subtle object state changes, while existing procedural datasets provide limited and inconsistent mistake and correction traces. We present PIE-V (Psychologically Inspired Error injection for Videos), a framework for constructing and benchmarking mistake-aware egocentric procedural videos by augmenting clean keystep procedures with controlled, human-plausible deviations. PIE-V combines a psychology-informed error planner conditioned on procedure phase and semantic step load, a correction planner that models recovery behavior, an LLM writer that performs cascade-consistent rewrites, and an LLM judge that validates procedural coherence and repairs failures. For video segment edits, PIE-V synthesizes replacement clips with text-guided video generation and stitches them into the episode to preserve visual plausibility. Applied to 17 tasks and 50 Ego-Exo4D scenarios, PIE-V injects 102 mistakes and generates 27 recovery corrections. For benchmarking, we introduce a unified taxonomy and a human rubric with nine metrics that cover step-level and procedure-level quality, including plausibility, procedure logic with annotator confidence, state change coherence, and grounding between text and video. Using this protocol, we audit several existing resources and compare PIE-V against a freeform LLM generation baseline under the same criteria. Together, the framework and rubric support post-completion verification for egocentric procedural mistake detection and correction.