How to Correctly Make Mistakes: A Framework for Constructing and Benchmarking Mistake Aware Egocentric Procedural Videos
Not on the map. Why?
Abstract (arXiv)
Reliable procedural monitoring in video requires exposure to naturally occurring human errors and the recoveries that follow. In egocentric recordings, mistakes are often partially occluded by hands and revealed through subtle object state changes, while existing procedural datasets provide limited and inconsistent mistake and correction traces. We present PIE-V (Psychologically Inspired Error injection for Videos), a framework for constructing and benchmarking mistake-aware egocentric procedural videos by augmenting clean keystep procedures with controlled, human-plausible deviations. PIE-V combines a psychology-informed error planner conditioned on procedure phase and semantic step load, a correction planner that models recovery behavior, an LLM writer that performs cascade-consistent rewrites, and an LLM judge that validates procedural coherence and repairs failures. For video segment edits, PIE-V synthesizes replacement clips with text-guided video generation and stitches them into the episode to preserve visual plausibility. Applied to 17 tasks and 50 Ego-Exo4D scenarios, PIE-V injects 102 mistakes and generates 27 recovery corrections. For benchmarking, we introduce a unified taxonomy and a human rubric with nine metrics that cover step-level and procedure-level quality, including plausibility, procedure logic with annotator confidence, state change coherence, and grounding between text and video. Using this protocol, we audit several existing resources and compare PIE-V against a freeform LLM generation baseline under the same criteria. Together, the framework and rubric support post-completion verification for egocentric procedural mistake detection and correction.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 17 of its 50 references that are in the map. First posted on arXiv 16 Apr 2026.
Builds on
17 of 50 references.
References · 17
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2023Qwen Technical Reportcited by 3,273
- 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 467
- 2022Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activitiescited by 284
- 2023HoloAssist: an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real Worldcited by 159
- 2024AnyV2V: A Tuning-Free Framework For Any Video-to-Video Editing Taskscited by 151
- 2022My View is the Best View: Procedure Learning from Egocentric Videoscited by 81
- 2023Learning Procedure-aware Video Representation from Instructional Videos and Their Narrationscited by 46
- 2023IndustReal: A Dataset for Procedure Step Recognition Handling Execution Errors in Egocentric Videos in an Industrial-Like Settingcited by 46
- 2024PREGO: online mistake detection in PRocedural EGOcentric videoscited by 36
- 2023Learning and Verification of Task Structure in Instructional Videoscited by 26
- 2023Every Mistake Counts in Assemblycited by 18
5 more
- 2025EasyV2V: A High-quality Instruction-based Video Editing Frameworkcited by 11
- 2024EgoOops: A Dataset for Mistake Action Detection from Egocentric Videos referring to Procedural Textscited by 5
- 2025Procedural Mistake Detection via Action Effect Modelingcited by 5
- 2024Transparent and Coherent Procedural Mistake Detectioncited by 4
- 2025Predicting Implicit Arguments in Procedural Video Instructionscited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.