# HEADS-UP: Head-Mounted Egocentric Dataset for Trajectory Prediction in Blind Assistance Systems

arXiv: https://arxiv.org/abs/2409.20324 · HTML: https://mlsanity.com/paper/2409.20324/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 26 references that are in the map.
- First posted on arXiv 30 Sep 2024.

## Builds on (most cited first, 14 of 14)

- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [ByteTrack: Multi-Object Tracking by Associating Every Detection Box](https://mlsanity.com/paper/2110.06864/) (2021)
- [Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks](https://mlsanity.com/paper/1803.10892/) (2018)
- [Stochastic Trajectory Prediction via Motion Indeterminacy Diffusion](https://mlsanity.com/paper/2203.13777/) (2022)
- [Transformer Networks for Trajectory Forecasting](https://mlsanity.com/paper/2003.08111/) (2020)
- [EgoBody: Human Body Shape and Motion of Interacting People from Head-Mounted Devices](https://mlsanity.com/paper/2112.07642/) (2021)
- [Human Trajectory Forecasting in Crowds: A Deep Learning Perspective](https://mlsanity.com/paper/2007.03639/) (2020)
- [Learning to Detect and Track Visible and Occluded Body Joints in a Virtual World](https://mlsanity.com/paper/1803.08319/) (2018)
- [You2Me: Inferring Body Pose in Egocentric Video via First and Second Person Interactions](https://mlsanity.com/paper/1904.09882/) (2019)
- [Mo2Cap2: Real-time Mobile 3D Motion Capture with a Cap-mounted Fisheye Camera](https://mlsanity.com/paper/1803.05959/) (2018)
- [EgoHumans: An Egocentric 3D Multi-Human Benchmark](https://mlsanity.com/paper/2305.16487/) (2023)
- [Social-Transmotion: Promptable Human Trajectory Prediction](https://mlsanity.com/paper/2312.16168/) (2023)
- [Unsupervised Sampling Promoting for Stochastic Human Trajectory Prediction](https://mlsanity.com/paper/2304.04298/) (2023)
- [VisionGPT: LLM-Assisted Real-Time Anomaly Detection for Safe Visual Navigation](https://mlsanity.com/paper/2403.12415/) (2024)

## Built on by (most cited first, 3 of 3)

- [Unified Human Localization and Trajectory Prediction with Monocular Vision](https://mlsanity.com/paper/2503.03535/) (2025)
- [Decoding Pedestrian Crossing Intention from Egocentric Vision via Vision Language Models](https://mlsanity.com/paper/2606.09142/) (2026)
- [Probabilistic Pedestrian Forecasts from a Handheld Phone: World-Frame Heat Maps, Visual-Inertial Height Drift, and Evaluation without Ground Truth](https://mlsanity.com/paper/2610.04736/) (2026)

## Abstract (arXiv)

In this paper, we introduce HEADS-UP, the first egocentric dataset collected from head-mounted cameras, designed specifically for trajectory prediction in blind assistance systems. With the growing population of blind and visually impaired individuals, the need for intelligent assistive tools that provide real-time warnings about potential collisions with dynamic obstacles is becoming critical. These systems rely on algorithms capable of predicting the trajectories of moving objects, such as pedestrians, to issue timely hazard alerts. However, existing datasets fail to capture the necessary information from the perspective of a blind individual. To address this gap, HEADS-UP offers a novel dataset focused on trajectory prediction in this context. Leveraging this dataset, we propose a semi-local trajectory prediction approach to assess collision risks between blind individuals and pedestrians in dynamic environments. Unlike conventional methods that separately predict the trajectories of both the blind individual (ego agent) and pedestrians, our approach operates within a semi-local coordinate system, a rotated version of the camera's coordinate system, facilitating the prediction process. We validate our method on the HEADS-UP dataset and implement the proposed solution in ROS, performing real-time tests on an NVIDIA Jetson GPU through a user study. Results from both dataset evaluations and live tests demonstrate the robustness and efficiency of our approach.