EgoIntent: A Pre-Outcome Micro-Step Benchmark for Understanding What, Why, and Next
Not on the map. Why?
Abstract (arXiv)
Egocentric video provides a natural modality for studying human behavior, but conventional visual understanding captures mainly observable scenes, objects, and actions rather than the latent goals that organize them. Existing intent benchmarks typically focus on coarse event-level goals and overlook how intent evolves across procedural steps. We introduce EgoIntent, a step-level intent-understanding benchmark comprising 3,014 pre-outcome micro-steps from 32 egocentric videos across 15 indoor and outdoor daily-life scenarios. Each step is manually annotated along three complementary dimensions: Local Intent (What), the immediate goal; Procedural Intent (Why), the role of the step in the broader procedure; and Next-Plan (Next), the action most likely to follow. Multiple rounds of human review refine temporal boundaries and annotation quality. We evaluate 15 multimodal large language models using reference-based scores and complementary reference-free diagnostics. Controlled studies on four representative models show that only one model gains significantly from correct temporal order, while a single boundary frame outperforms the full ordered clip for three models. Step-only input performs best for all four models, and adding 15 seconds of history significantly degrades three. Revealing the current outcome improves Local Intent by 7.81 points, while revealing the following step improves Next-Plan by 13.17 points. These findings indicate that current models can achieve strong intent-prediction scores through static boundary cues without robustly exploiting temporal order or procedural history.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 35 of its 63 references that are in the map. First posted on arXiv 12 Mar 2026.
Builds on
35 of 63 references.
References · 35
- 2025Qwen3 Technical Reportcited by 8,195
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
- 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
- 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
- 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 789
- 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 735
- 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 666
- 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 559
- 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 550
23 more
- 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 477
- 2026Kimi K2.5: Visual Agentic Intelligencecited by 449
- 2025Kimi-VL Technical Reportcited by 381
- 2022Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activitiescited by 284
- 2026Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Groundingcited by 132
- 2018Charades-Ego: A Large-Scale Dataset of Paired Third and First Person Videoscited by 127
- 2025HD-EPIC: A Highly-Detailed Egocentric Video Datasetcited by 101
- 2024Proactive Agent: Shifting LLM Agents from Reactive Responses to Active Assistancecited by 84
- 2023EgoPlan-Bench: Benchmarking Multimodal Large Language Models for Human-Level Planningcited by 66
- 2024VidEgoThink: Assessing Egocentric Video Understanding Capabilities for Embodied AIcited by 30
- 2024ObjectRelator: Enabling Cross-View Object Relation Understanding Across Ego-Centric and Exo-Centric Perspectivescited by 26
- 2025In the Eye of MLLM: Benchmarking Egocentric Video Intent Understanding with Gaze-Guided Promptingcited by 21
- 2025AI for Service: Proactive Assistance with AI Glassescited by 18
- 2025EgoNight: Towards Egocentric Vision Understanding at Night with a Challenging Benchmarkcited by 16
- 2025X-LeBench: A Benchmark for Extremely Long Egocentric Video Understandingcited by 14
- 2025Sensible Agent: A Framework for Unobtrusive Interaction with Proactive AR Agentscited by 14
- 2025Exo2EgoSyn: Unlocking Foundation Video Generation Models for Exocentric-to-Egocentric Video Synthesiscited by 12
- 2025EgoVLM: Policy Optimization for Egocentric Video Understandingcited by 11
- 2025Intention-Guided Cognitive Reasoning for Egocentric Long-Term Action Anticipationcited by 10
- 2025Benchmarking Egocentric Multimodal Goal Inference for Assistive Wearable Agentscited by 8
- 2026EgoSound: Benchmarking Sound Understanding in Egocentric Videoscited by 8
- 2025HiERO: understanding the hierarchy of human behavior enhances reasoning on egocentric videoscited by 7
- 2025EgoVITA: Learning to Plan and Verify for Egocentric Video Reasoningcited by 4
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2022EgoTaskQA: Understanding Human Tasks in Egocentric Videos
- 2024EAGLE: Egocentric AGgregated Language-video Engine
- 2026Ego2Act: Evaluating Goal-Directed Manipulation in Egocentric Video Generation
- 2026EgoExo-Next:Benchmarking Vision-Language Models on Visual-Option Next-State and Cross-View Reasoning
- 2026EXPLORE-Bench: Egocentric Scene Prediction with Long-Horizon Reasoning
- 2026EgoMonth: A Month-Level Egocentric Video Benchmark for Long-Term Spatiotemporal Memory
- 2026Does Local Video Understanding Transfer Across Encounters? The EgoGears Benchmark
- 2026EgoMemReason: A Memory-Driven Reasoning Benchmark for Long-Horizon Egocentric Video Understanding