# LookOut: Real-World Humanoid Egocentric Navigation

arXiv: https://arxiv.org/abs/2508.14466 · HTML: https://mlsanity.com/paper/2508.14466/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 55 of its 69 references that are in the map.
- First posted on arXiv 20 Aug 2025.

## Builds on (most cited first, 30 of 55)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D](https://mlsanity.com/paper/2008.05711/) (2020)
- [Scaling Egocentric Vision: The EPIC-KITCHENS Dataset](https://mlsanity.com/paper/1804.02748/) (2018)
- [DeepVoxels: Learning Persistent 3D Feature Embeddings](https://mlsanity.com/paper/1812.01024/) (2018)
- [Depth Pro: Sharp Monocular Metric Depth in Less Than a Second](https://mlsanity.com/paper/2410.02073/) (2024)
- [Decomposing NeRF for Editing via Feature Field Distillation](https://mlsanity.com/paper/2205.15585/) (2022)
- [REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environments](https://mlsanity.com/paper/1904.10151/) (2019)
- [HuMoR: 3D Human Motion Model for Robust Pose Estimation](https://mlsanity.com/paper/2105.04668/) (2021)
- [NaVILA: Legged Robot Vision-Language-Action Model for Navigation](https://mlsanity.com/paper/2412.04453/) (2024)
- [Project Aria: A New Tool for Egocentric Multi-Modal AI Research](https://mlsanity.com/paper/2308.13561/) (2023)
- [Neural Feature Fusion Fields: 3D Distillation of Self-Supervised 2D Image Representations](https://mlsanity.com/paper/2209.03494/) (2022)
- [CoWs on Pasture: Baselines and Benchmarks for Language-Driven Zero-Shot Object Navigation](https://mlsanity.com/paper/2203.10421/) (2022)
- [Simple-BEV: What Really Matters for Multi-Sensor BEV Perception?](https://mlsanity.com/paper/2206.07959/) (2022)
- [Socially Compliant Navigation Dataset (SCAND): A Large-Scale Dataset of Demonstrations for Social Navigation](https://mlsanity.com/paper/2203.15041/) (2022)
- [Nymeria: A Massive Collection of Multimodal Egocentric Daily Motion in the Wild](https://mlsanity.com/paper/2406.09905/) (2024)
- [Actor and Observer: Joint Modeling of First and Third-Person Videos](https://mlsanity.com/paper/1804.09627/) (2018)
- [Ego-Exo: Transferring Visual Representations from Third-person to First-person Videos](https://mlsanity.com/paper/2104.07905/) (2021)
- [Ego-Body Pose Estimation via Ego-Head Pose Estimation](https://mlsanity.com/paper/2212.04636/) (2022)
- [GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigation](https://mlsanity.com/paper/2404.06609/) (2024)
- [Ego-Pose Estimation and Forecasting as Real-Time PD Control](https://mlsanity.com/paper/1906.03173/) (2019)
- [You2Me: Inferring Body Pose in Egocentric Video via First and Second Person Interactions](https://mlsanity.com/paper/1904.09882/) (2019)
- [xR-EgoPose: Egocentric 3D Human Pose from an HMD Camera](https://mlsanity.com/paper/1907.10045/) (2019)
- [HM3D-OVON: A Dataset and Benchmark for Open-Vocabulary Object Goal Navigation](https://mlsanity.com/paper/2409.14296/) (2024)
- [Humanoid Locomotion as Next Token Prediction](https://mlsanity.com/paper/2402.19469/) (2024)
- [DINO-Tracker: Taming DINO for Self-Supervised Point Tracking in a Single Video](https://mlsanity.com/paper/2403.14548/) (2024)

## Built on by (most cited first, 9 of 9)

- [EgoReAct: Egocentric Video-Driven 3D Human Reaction Generation](https://mlsanity.com/paper/2512.22808/) (2025)
- [ECHO: Ego-Centric modeling of Human-Object interactions](https://mlsanity.com/paper/2508.21556/) (2025)
- [MM-Nav: Multi-View VLA Model for Robust Visual Navigation via Multi-Expert Learning](https://mlsanity.com/paper/2510.03142/) (2025)
- [EgoCogNav: Cognition-aware Human Egocentric Navigation](https://mlsanity.com/paper/2511.17581/) (2025)
- [Mobile UMI: Cross-View Diffusion Policy with Decoupled Kinematics for Mobile Manipulation](https://mlsanity.com/paper/2605.20894/) (2026)
- [EgoTraj: Real-World Egocentric Human Trajectory Dataset for Multimodal Prediction](https://mlsanity.com/paper/2605.19004/) (2026)
- [Decoding Pedestrian Crossing Intention from Egocentric Vision via Vision Language Models](https://mlsanity.com/paper/2606.09142/) (2026)
- [Learning Humanoid Navigation from Human Data](https://mlsanity.com/paper/2604.00416/) (2026)
- [EgoMotion: Hierarchical Reasoning and Diffusion for Egocentric Vision-Language Motion Generation](https://mlsanity.com/paper/2604.19105/) (2026)

## Abstract (arXiv)

The ability to predict collision-free future trajectories from egocentric observations is crucial in applications such as humanoid robotics, VR / AR, and assistive navigation. In this work, we introduce the challenging problem of predicting a sequence of future 6D head poses from an egocentric video. In particular, we predict both head translations and rotations to learn the active information-gathering behavior expressed through head-turning events. To solve this task, we propose a framework that reasons over temporally aggregated 3D latent features, which models the geometric and semantic constraints for both the static and dynamic parts of the environment. Motivated by the lack of training data in this space, we further contribute a data collection pipeline using the Project Aria glasses, and present a dataset collected through this approach. Our dataset, dubbed Aria Navigation Dataset (AND), consists of 4 hours of recording of users navigating in real-world scenarios. It includes diverse situations and navigation behaviors, providing a valuable resource for learning real-world egocentric navigation policies. Extensive experiments show that our model learns human-like navigation behaviors such as waiting / slowing down, rerouting, and looking around for traffic while generalizing to unseen environments. Check out our project webpage at https://sites.google.com/stanford.edu/lookout.