mlsanityData to 14 Oct 2026

EgoThinker: Unveiling Egocentric Reasoning with Spatio-Temporal CoT

arXiv 2510.23569 ↗First posted 27 Oct 2025cs.CVcited by 16

On the map: LLM reasoning → Spatial and embodied reasoning

Abstract (arXiv)

Egocentric video reasoning centers on an unobservable agent behind the camera who dynamically shapes the environment, requiring inference of hidden intentions and recognition of fine-grained interactions. This core challenge limits current multimodal large language models MLLMs, which excel at visible event reasoning but lack embodied, first-person understanding. To bridge this gap, we introduce EgoThinker, a novel framework that endows MLLMs with robust egocentric reasoning capabilities through spatio-temporal chain-of-thought supervision and a two-stage learning curriculum. First, we introduce EgoRe-5M, a large-scale egocentric QA dataset constructed from 13M diverse egocentric video clips. This dataset features multi-minute segments annotated with detailed CoT rationales and dense hand-object grounding. Second, we employ SFT on EgoRe-5M to instill reasoning skills, followed by reinforcement fine-tuning RFT to further enhance spatio-temporal localization. Experimental results show that EgoThinker outperforms existing methods across multiple egocentric benchmarks, while achieving substantial improvements in fine-grained spatio-temporal localization tasks. Full code and data are released at https://github.com/InternRobotics/EgoThinker.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 63 of its 112 references that are in the map. First posted on arXiv 27 Oct 2025.

Builds on

63 of 112 references.

References · 63

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  3. 2025Qwen2.5-VL Technical Reportcited by 5,587
  4. 2024GPT-4o System Cardcited by 4,588
  5. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  6. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  7. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  8. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  9. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
  10. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  11. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
  12. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
38 more
  1. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
  2. 2024OpenAI o1 System Cardcited by 1,244
  3. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
  4. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,062
  5. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 1,053
  6. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,050
  7. 2023VideoChat: Chat-Centric Video Understandingcited by 857
  8. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 723
  9. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 688
  10. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 583
  11. 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 518
  12. 2024Long Context Transfer from Language to Visioncited by 515
  13. 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 513
  14. 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 497
  15. 2019CLEVRER: CoLlision Events for Video REpresentation and Reasoningcited by 484
  16. 2018TVQA: Localized, Compositional Video Question Answeringcited by 467
  17. 2025R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalizationcited by 370
  18. 2018A Short Note about Kinetics-600cited by 333
  19. 2024InternVideo2: Scaling Foundation Models for Multimodal Video Understandingcited by 317
  20. 2025R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimizationcited by 301
  21. 2025VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuningcited by 255
  22. 2025LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RLcited by 248
  23. 2025Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcementcited by 239
  24. 2025InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modelingcited by 211
  25. 2025R1-Zero's "Aha Moment" in Visual Reasoning on a 2B Non-SFT Modelcited by 165
  26. 2024M3CoT: A Novel Benchmark for Multi-Domain Multi-step Multi-modal Chain-of-Thoughtcited by 121
  27. 2025Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer Modelscited by 100
  28. 2025EgoLife: Towards Egocentric Life Assistantcited by 91
  29. 2024VideoHallucer: Evaluating Intrinsic and Extrinsic Hallucinations in Large Video-Language Modelscited by 90
  30. 2024CG-Bench: Clue-grounded Question Answering Benchmark for Long Video Understandingcited by 85
  31. 2025Gemini Robotics: Bringing AI into the Physical Worldcited by 83
  32. 2025Boosting the Generalization and Reasoning of Vision Language Models with Curriculum Reinforcement Learningcited by 71
  33. 2025Eagle 2: Building Post-Training Data Strategies from Scratch for Frontier Vision-Language Modelscited by 70
  34. 2025Vision-R1: Evolving Human-Free Alignment in Large Vision-Language Models via Vision-Guided Reinforcement Learningcited by 69
  35. 2025Eagle 2.5: Boosting Long-Context Post-Training for Frontier Vision-Language Modelscited by 64
  36. 2023LifelongMemory: Leveraging LLMs for Answering Queries in Long-form Egocentric Videoscited by 51
  37. 2025STORM: Token-Efficient Long Video Understanding for Multimodal LLMscited by 49
  38. 2022InternVideo-Ego4D: A Pack of Champion Solutions to Ego4D Challengescited by 48

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 13.

16 papers.

Sort:

By area:

Similar, not cited

Close in wording, no citation either way

  1. 2026Minerva-Ego: Spatiotemporal Hints for Egocentric Video Understanding
  2. 2026EgoCoT-Bench: Benchmarking Grounded and Verifiable Operation-Centric Chain of Thought Reasoning for MLLMs
  3. 2025EgoVLM: Policy Optimization for Egocentric Video Understanding
  4. 2025EgoToM: Benchmarking Theory of Mind Reasoning from Egocentric Videos
  5. 2025Omnia de EgoTempo: Benchmarking Temporal Understanding of Multi-Modal LLMs in Egocentric Videos
  6. 2026From Perception to Planning: Evolving Ego-Centric Task-Oriented Spatiotemporal Reasoning via Curriculum Learning
  7. 2026Spatial-Conditioned Reasoning in Long-Egocentric Videos
  8. 2026EgoSAT: A Comprehensive Benchmark of Egocentric Streaming Interaction Understanding