mlsanityData to 14 Oct 2026

Technical Report for Egocentric Mistake Detection for the HoloAssist Challenge

arXiv 2506.06174 ↗First posted 6 Jun 2025cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

In this report, we address the task of online mistake detection, which is vital in domains like industrial automation and education, where real-time video analysis allows human operators to correct errors as they occur. While previous work focuses on procedural errors involving action order, broader error types must be addressed for real-world use. We introduce an online mistake detection framework that handles both procedural and execution errors (e.g., motor slips or tool misuse). Upon detecting an error, we use a large language model (LLM) to generate explanatory feedback. Experiments on the HoloAssist benchmark confirm the effectiveness of our approach, where our approach is placed second on the mistake detection task.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 13 of its 16 references that are in the map. First posted on arXiv 6 Jun 2025.

Builds on

13 of 16 references.

References · 13

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  5. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,620
  6. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
  7. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,062
  8. 2023HoloAssist: an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real Worldcited by 166
  9. 2021OadTR: Online Action Detection with Transformerscited by 83
  10. 2024PREGO: online mistake detection in PRocedural EGOcentric videoscited by 39
  11. 2024Differentiable Task Graph Learning: Procedural Activity Representation and Online Mistake Detection from Egocentric Videoscited by 26
  12. 2022Colar: Effective and Efficient Online Action Detection by Consulting Exemplarscited by 21
1 more
  1. 2023ZEETAD: Adapting Pretrained Vision-Language Model for Zero-Shot End-to-End Temporal Action Detectioncited by 12

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.