# EgoTraj: Real-World Egocentric Human Trajectory Dataset for Multimodal Prediction

arXiv: https://arxiv.org/abs/2605.19004 · HTML: https://mlsanity.com/paper/2605.19004/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 44 references that are in the map.
- First posted on arXiv 18 May 2026.

## Builds on (most cited first, 26 of 26)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [Depth Anything V2](https://mlsanity.com/paper/2406.09414/) (2024)
- [Planning-oriented Autonomous Driving](https://mlsanity.com/paper/2212.10156/) (2022)
- [Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset](https://mlsanity.com/paper/2104.10133/) (2021)
- [Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives](https://mlsanity.com/paper/2311.18259/) (2023)
- [OneFormer: One Transformer to Rule Universal Image Segmentation](https://mlsanity.com/paper/2211.06220/) (2022)
- [GenAD: Generative End-to-End Autonomous Driving](https://mlsanity.com/paper/2402.11502/) (2024)
- [Crowd-Robot Interaction: Crowd-aware Robot Navigation with Attention-based Deep Reinforcement Learning](https://mlsanity.com/paper/1809.08835/) (2018)
- [Aria Digital Twin: A New Benchmark Dataset for Egocentric 3D Machine Perception](https://mlsanity.com/paper/2306.06362/) (2023)
- [The inD Dataset: A Drone Dataset of Naturalistic Road User Trajectories at German Intersections](https://mlsanity.com/paper/1911.07602/) (2019)
- [Socially Compliant Navigation Dataset (SCAND): A Large-Scale Dataset of Demonstrations for Social Navigation](https://mlsanity.com/paper/2203.15041/) (2022)
- [Nymeria: A Massive Collection of Multimodal Egocentric Daily Motion in the Wild](https://mlsanity.com/paper/2406.09905/) (2024)
- [Future Person Localization in First-Person Videos](https://mlsanity.com/paper/1711.11217/) (2017)
- [Social-Implicit: Rethinking Trajectory Prediction Evaluation and The Effectiveness of Implicit Maximum Likelihood Estimation](https://mlsanity.com/paper/2203.03057/) (2022)
- [Discrete Residual Flow for Probabilistic Pedestrian Behavior Prediction](https://mlsanity.com/paper/1910.08041/) (2019)
- [Toward Human-Like Social Robot Navigation: A Large-Scale, Multi-Modal, Social Human Navigation Dataset](https://mlsanity.com/paper/2303.14880/) (2023)
- [Egocentric Human Trajectory Forecasting with a Wearable Camera and Multi-Modal Fusion](https://mlsanity.com/paper/2111.00993/) (2021)
- [EgoCast: Forecasting Egocentric Human Pose in the Wild](https://mlsanity.com/paper/2412.02903/) (2024)
- [EgoNav: Egocentric Scene-aware Human Trajectory Prediction](https://mlsanity.com/paper/2403.19026/) (2024)
- [LookOut: Real-World Humanoid Egocentric Navigation](https://mlsanity.com/paper/2508.14466/) (2025)
- [Real-Time Safe Bipedal Robot Navigation using Linear Discrete Control Barrier Functions](https://mlsanity.com/paper/2411.03619/) (2024)
- [ARMOR: Egocentric Perception for Humanoid Robot Collision Avoidance and Motion Planning](https://mlsanity.com/paper/2412.00396/) (2024)
- [EgoCogNav: Cognition-aware Human Egocentric Navigation](https://mlsanity.com/paper/2511.17581/) (2025)
- [ARCAS: An Augmented Reality Collision Avoidance System with SLAM-Based Tracking for Enhancing VRU Safety](https://mlsanity.com/paper/2512.05299/) (2025)

## Built on by (most cited first, 1 of 1)

- [Human-Centric Intelligence in the Era of Foundation Models: A Survey](https://mlsanity.com/paper/2608.18184/) (2026)

## Abstract (arXiv)

Accurately forecasting human trajectories from an egocentric perspective plays a central role in applications such as humanoid robotics, wearable sensing systems, and assistive navigation. However, progress in this direction remains limited due to the scarcity of egocentric trajectory datasets collected in real-world environments. Addressing this need, we introduce EgoTraj, an egocentric multimodal open dataset recorded using Meta Quest Pro (MQPro). EgoTraj contains 75 sequences of human navigation collected from multiple MQPro wearers in real-world urban environments. Each recording provides synchronized RGB video along with ground-truth data, including continuous time-synchronized 6-degree-of-freedom head poses, per-frame 3D eye gaze vectors, scene annotations. To the best of our knowledge, EgoTraj differs from typical egocentric trajectory datasets by capturing long-horizon, self-directed navigation across diverse urban routes with broad participant diversity. To demonstrate the potential of the dataset, we benchmark several state-of-the-art methods for egocentric trajectory prediction and conduct ablation studies to analyze the contributions of gaze, scene, and motion cues. The results highlight the utility of EgoTraj for AR-based perception, navigation, and assistive systems. The EgoTraj dataset, code, and EgoViz Dashboard are publicly available at https://github.com/yehiahmad/EgoTraj.