# EgoCogNav: Cognition-aware Human Egocentric Navigation

arXiv: https://arxiv.org/abs/2511.17581 · HTML: https://mlsanity.com/paper/2511.17581/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 72 references that are in the map.
- First posted on arXiv 15 Nov 2025.

## Builds on (most cited first, 30 of 40)

- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning](https://mlsanity.com/paper/1702.03118/) (2017)
- [Scaling Egocentric Vision: The EPIC-KITCHENS Dataset](https://mlsanity.com/paper/1804.02748/) (2018)
- [ReMoDiffuse: Retrieval-Augmented Motion Diffusion Model](https://mlsanity.com/paper/2304.01116/) (2023)
- [Project Aria: A New Tool for Egocentric Multi-Modal AI Research](https://mlsanity.com/paper/2308.13561/) (2023)
- [Stochastic Trajectory Prediction via Motion Indeterminacy Diffusion](https://mlsanity.com/paper/2203.13777/) (2022)
- [Leapfrog Diffusion Model for Stochastic Trajectory Prediction](https://mlsanity.com/paper/2303.10895/) (2023)
- [Human Trajectory Forecasting in Crowds: A Deep Learning Perspective](https://mlsanity.com/paper/2007.03639/) (2020)
- [HOT3D: Hand and Object Tracking in 3D from Egocentric Multi-View Videos](https://mlsanity.com/paper/2411.19167/) (2024)
- [Nymeria: A Massive Collection of Multimodal Egocentric Daily Motion in the Wild](https://mlsanity.com/paper/2406.09905/) (2024)
- [Core Challenges of Social Robot Navigation: A Survey](https://mlsanity.com/paper/2103.05668/) (2021)
- [Ego-Exo: Transferring Visual Representations from Third-person to First-person Videos](https://mlsanity.com/paper/2104.07905/) (2021)
- [Ego-Body Pose Estimation via Ego-Head Pose Estimation](https://mlsanity.com/paper/2212.04636/) (2022)
- [Remember Intentions: Retrospective-Memory-based Trajectory Prediction](https://mlsanity.com/paper/2203.11474/) (2022)
- [xR-EgoPose: Egocentric 3D Human Pose from an HMD Camera](https://mlsanity.com/paper/1907.10045/) (2019)
- [In the Eye of the Beholder: Gaze and Actions in First Person Video](https://mlsanity.com/paper/2006.00626/) (2020)
- [SACSoN: Scalable Autonomous Control for Social Navigation](https://mlsanity.com/paper/2306.01874/) (2023)
- [Estimating Body and Hand Motion in an Ego-sensed World](https://mlsanity.com/paper/2410.03665/) (2024)
- [VLM-Social-Nav: Socially Aware Robot Navigation through Scoring using Vision-Language Models](https://mlsanity.com/paper/2404.00210/) (2024)
- [Discount Factor as a Regularizer in Reinforcement Learning](https://mlsanity.com/paper/2007.02040/) (2020)
- [Aria Everyday Activities Dataset](https://mlsanity.com/paper/2402.13349/) (2024)
- [EgoExo-Fitness: Towards Egocentric and Exocentric Full-Body Action Understanding](https://mlsanity.com/paper/2406.08877/) (2024)
- [SingularTrajectory: Universal Trajectory Predictor Using Diffusion Model](https://mlsanity.com/paper/2403.18452/) (2024)
- [Discrete Residual Flow for Probabilistic Pedestrian Behavior Prediction](https://mlsanity.com/paper/1910.08041/) (2019)
- [Social-Transmotion: Promptable Human Trajectory Prediction](https://mlsanity.com/paper/2312.16168/) (2023)
- [Toward Human-Like Social Robot Navigation: A Large-Scale, Multi-Modal, Social Human Navigation Dataset](https://mlsanity.com/paper/2303.14880/) (2023)

## Built on by (most cited first, 3 of 3)

- [EgoTraj: Real-World Egocentric Human Trajectory Dataset for Multimodal Prediction](https://mlsanity.com/paper/2605.19004/) (2026)
- [Decoding Pedestrian Crossing Intention from Egocentric Vision via Vision Language Models](https://mlsanity.com/paper/2606.09142/) (2026)
- [Learning Humanoid Navigation from Human Data](https://mlsanity.com/paper/2604.00416/) (2026)

## Abstract (arXiv)

Modeling the cognitive and experiential factors of human navigation is central to deepening our understanding of human-environment interaction and to enabling safe social navigation and effective assistive wayfinding. Most existing methods focus on forecasting motions in fully observed scenes and often neglect human factors that capture how people feel and respond to space. To address this gap, we propose EgoCogNav, a multimodal egocentric navigation framework that jointly forecasts perceived path uncertainty, trajectories and head motion from egocentric video, gaze, and motion history. To facilitate research in the field, we introduce the Cognition-aware Egocentric Navigation (CEN) dataset consisting of 6 hours real-world egocentric recordings capturing diverse navigation behaviors in real-world scenarios. Experiments show that EgoCogNav learns the perceived uncertainty that strongly correlates with human-like behaviors such as scanning, hesitation, and backtracking while improving trajectory and head-motion forecasting on held-out navigation recordings.