# RegulAR: Graph-Grounded Error Recognition and Assistance for Procedural Tasks in AR

arXiv: https://arxiv.org/abs/2608.26715 · HTML: https://mlsanity.com/paper/2608.26715/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Aug 2026.

## Built on by (most cited first, 1 of 1)

- [Generative Tutorial: Towards Live Contextualized Visual Instructions for Physical Tasks](https://mlsanity.com/paper/2609.24955/) (2026)

## Abstract (arXiv)

Errors are inevitable in procedural tasks, yet most AR guidance systems focus on step-by-step instruction delivery rather than helping users recognize and recover from mistakes. We present RegulAR, an AR task assistant for procedural error recognition and recovery. RegulAR models task instructions as a hierarchical dependency graph and combines this structure with a Multimodal Large Language Model (MLLM) to interpret egocentric observations during execution. This enables RegulAR to track progress, identify deviations by error type, estimate their impact on later steps, and deliver appropriately salient interventions through an in-situ head-up display that visualizes task state and recovery guidance. By making procedural structure explicit, RegulAR supports not only next-step guidance, but also reasoning about what went wrong, why it matters, and how users can get back on track. In a within-subject study (N=12), participants reported better task-structure understanding and recovery support with RegulAR than the MLLM-only baseline.