EgoVITA: Learning to Plan and Verify for Egocentric Video Reasoning
On the map: LLM reasoning → Spatial and embodied reasoning
Abstract (arXiv)
Egocentric video understanding requires procedural reasoning under partial observability and continuously shifting viewpoints. Current multimodal large language models (MLLMs) struggle with this setting, often generating plausible but visually inconsistent or weakly grounded responses. We introduce EgoVITA, a framework that decomposes egocentric video reasoning into a structured textitplan-then-verify process. The model first generates an egocentric plan: a causal sequence of anticipated actions from a first-person perspective. This plan is then evaluated by an exocentric verification stage that uses third-person reasoning over the same video to verify its spatiotemporal and logical consistency, without exocentric video input. This decomposition enables cross-perspective feedback without requiring paired ego-exo supervision. To train this reasoning process, we adopt Group Relative Policy Optimization (GRPO) with two dense reward signals: one that grounds anticipated actions in subsequent visual observations and another that reinforces consistent third-person verification. EgoVITA achieves state-of-the-art performance on egocentric reasoning benchmarks, outperforming Qwen2.5-VL-7B by +7.7 on EgoBlind and +4.4 on EgoOrient, while maintaining strong generalization on exocentric video tasks with only 52k training samples.
Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 50 of its 61 references that are in the map. First posted on arXiv 23 Nov 2025.
Builds on
50 of 61 references.
Closest ancestors · 8
Same direction, newest first
- 2025Actial: Activate Spatial Reasoning Ability of Multimodal Large Language Models3 Nov
- 2025EgoThinker: Unveiling Egocentric Reasoning with Spatio-Temporal CoT27 Oct
- 2025Spatial Reasoning with Vision-Language Models in Ego-Centric Multi-View Scenes8 Sep
- 2025Ego-R1: Chain-of-Tool-Thought for Ultra-Long Egocentric Video Reasoning16 Jun
- 2025EgoVLM: Policy Optimization for Egocentric Video Understanding3 Jun
- 2025Perspective-Aware Reasoning in Vision-Language Models via Mental Imagery Simulation24 Apr
- 2025EgoToM: Benchmarking Theory of Mind Reasoning from Egocentric Videos28 Mar
- 2023EgoPlan-Bench: Benchmarking Multimodal Large Language Models for Human-Level Planning11 Dec
Other references · 42
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024GPT-4o System Cardcited by 4,588
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
- 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
- 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
- 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
- 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,109
- 2024DoRA: Weight-Decomposed Low-Rank Adaptationcited by 758
30 more
- 2024LVBench: An Extreme Long Video Understanding Benchmarkcited by 410
- 2025Seed1.5-VL Technical Reportcited by 373
- 2024SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuningcited by 355
- 2025HD-EPIC: A Highly-Detailed Egocentric Video Datasetcited by 104
- 2025EgoLife: Towards Egocentric Life Assistantcited by 91
- 2022My View is the Best View: Procedure Learning from Egocentric Videoscited by 81
- 2024TOMATO: Assessing Visual Temporal Reasoning Capabilities in Multimodal Foundation Modelscited by 73
- 2023EgoThink: Evaluating First-Person Perspective Thinking Capability of Vision-Language Modelscited by 60
- 2025EgoTextVQA: Towards Egocentric Scene-Text Aware Video Question Answeringcited by 44
- 2025Omnia de EgoTempo: Benchmarking Temporal Understanding of Multi-Modal LLMs in Egocentric Videoscited by 44
- 2023WEAR: An Outdoor Sports Dataset for Wearable and Egocentric Activity Recognitioncited by 32
- 2024Open-ended VQA benchmarking of Vision-Language models by exploiting Classification datasets and their semantic hierarchycited by 26
- 2025Exo2Ego: Exocentric Knowledge Guided MLLM for Egocentric Video Understandingcited by 25
- 2025EgoExoBench: A Benchmark for First- and Third-person View Video Understanding in MLLMscited by 23
- 2025In the Eye of MLLM: Benchmarking Egocentric Video Intent Understanding with Gaze-Guided Promptingcited by 22
- 2025EgoBlind: Towards Egocentric Visual Assistance for the Blindcited by 21
- 2025EOC-Bench: Can MLLMs Identify, Recall, and Forecast Objects in an Egocentric World?cited by 21
- 2025EgoExo-Gen: Ego-centric Video Prediction by Watching Exo-centric Videoscited by 19
- 2025Where MLLMs Attend and What They Rely On: Explaining Autoregressive Token Generationcited by 16
- 2025AVATAR: Reinforcement Learning to See, Hear, and Reason Over Videocited by 12
- 2025Visual Intention Grounding for Egocentric Assistantscited by 11
- 2025Fine-grained Spatiotemporal Grounding on Egocentric Videoscited by 11
- 2025GuideDog: A Real-World Egocentric Multimodal Dataset for Blind and Low-Vision Accessibility-Aware Guidancecited by 10
- 2025EGOILLUSION: Benchmarking Hallucinations in Egocentric Video Understandingcited by 10
- 2024Is 'Right' Right? Enhancing Object Orientation Understanding in Multimodal Large Language Models through Egocentric Instruction Tuningcited by 8
- 2025VideoPASTA: 7K Preference Pairs That Matter for Video-LLM Alignmentcited by 8
- 2024VideoSAVi: Self-Aligned Video Language Models without Human Supervisioncited by 7
- 2025O-MaMa: Learning Object Mask Matching between Egocentric and Exocentric Viewscited by 7
- 2025EgoExo-Con: Exploring View-Invariant Video Temporal Understandingcited by 5
- 2025Let Me Show You: Learning by Retrieving from Egocentric Video for Robotic Manipulationcited by 2
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026EgoExoMem: Cross-View Memory Reasoning over Synchronized Egocentric and Exocentric Videoscited by 3
- 2026EgoIntent: A Pre-Outcome Micro-Step Benchmark for Understanding What, Why, and Nextcited by 1
- 2026Incentivizing Temporal-Awareness in Egocentric Video Understanding Modelscited by 0
- 2026Video-FLAIR: Not Whether to Reason, But Howcited by 0
- 2026Video-FLAIR: Not Whether to Reason, But How27 Aug 2026
- 2026EgoExoMem: Cross-View Memory Reasoning over Synchronized Egocentric and Exocentric Videos18 May 2026
- 2026Incentivizing Temporal-Awareness in Egocentric Video Understanding Models28 Mar 2026
- 2026EgoIntent: A Pre-Outcome Micro-Step Benchmark for Understanding What, Why, and Next12 Mar 2026
Similar, not cited
Close in wording, no citation either way
- 2026EgoCoT-Bench: Benchmarking Grounded and Verifiable Operation-Centric Chain of Thought Reasoning for MLLMs
- 2026EgoReasoner: Learning Egocentric 4D Reasoning via Task-Adaptive Structured Thinking
- 2026Minerva-Ego: Spatiotemporal Hints for Egocentric Video Understanding
- 2025From My View to Yours: Learning Egocentric Cues from Exocentric Video using Privileged Egocentric Supervision
- 2026Spatial-Conditioned Reasoning in Long-Egocentric Videos
- 2026OmniEgo-R2: A Routed Reasoning Framework for the 1st Cross-Domain EgoCross Challenge at CVPR 2026
- 2026EgoExo-Next:Benchmarking Vision-Language Models on Visual-Option Next-State and Cross-View Reasoning
- 2026EXPLORE-Bench: Egocentric Scene Prediction with Long-Horizon Reasoning