# Egocentric Event-Based Vision for Ping Pong Ball Trajectory Prediction

arXiv: https://arxiv.org/abs/2506.07860 · HTML: https://mlsanity.com/paper/2506.07860/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 41 references that are in the map.
- First posted on arXiv 9 Jun 2025.

## Builds on (most cited first, 19 of 19)

- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [Project Aria: A New Tool for Egocentric Multi-Modal AI Research](https://mlsanity.com/paper/2308.13561/) (2023)
- [Event-based Moving Object Detection and Tracking](https://mlsanity.com/paper/1803.04523/) (2018)
- [TTNet: Real-time temporal and spatial video analysis of table tennis](https://mlsanity.com/paper/2004.09927/) (2020)
- [Achieving Human Level Competitive Robot Table Tennis](https://mlsanity.com/paper/2408.03906/) (2024)
- [Event-based Agile Object Catching with a Quadrupedal Robot](https://mlsanity.com/paper/2303.17479/) (2023)
- [Strategy and Skill Learning for Physics-based Table Tennis Animation](https://mlsanity.com/paper/2407.16210/) (2024)
- [EV-Catcher: High-Speed Object Catching Using Low-latency Event-based Neural Networks](https://mlsanity.com/paper/2304.07200/) (2023)
- [Robotic Table Tennis: A Case Study into a High Speed Learning System](https://mlsanity.com/paper/2309.03315/) (2023)
- [Real Time Trajectory Prediction Using Deep Conditional Generative Models](https://mlsanity.com/paper/1909.03895/) (2019)
- [Table tennis ball spin estimation with an event camera](https://mlsanity.com/paper/2404.09870/) (2024)
- [Spin Detection in Robotic Table Tennis](https://mlsanity.com/paper/1905.07967/) (2019)
- [SpinDOE: A ball spin estimation method for table tennis robot](https://mlsanity.com/paper/2303.03879/) (2023)
- [Imaging for All-Day Wearable Smart Glasses](https://mlsanity.com/paper/2504.13060/) (2025)
- [Fast Trajectory End-Point Prediction with Event Cameras for Reactive Robot Control](https://mlsanity.com/paper/2302.13796/) (2023)
- [SpikeMOT: Event-based Multi-Object Tracking with Sparse Motion Features](https://mlsanity.com/paper/2309.16987/) (2023)
- [An Event-Based Perception Pipeline for a Table Tennis Robot](https://mlsanity.com/paper/2502.00749/) (2025)
- [Black-Box vs. Gray-Box: A Case Study on Learning Table Tennis Ball Trajectory Prediction with Spin and Impacts](https://mlsanity.com/paper/2305.15189/) (2023)
- [avaTTAR: Table Tennis Stroke Training with On-body and Detached Visualization in Augmented Reality](https://mlsanity.com/paper/2407.15373/) (2024)

## Built on by (most cited first, 3 of 3)

- [BilliardPhys-Bench: Benchmarking Physical Reasoning and Visual Dynamics of Multimodal LLMs](https://mlsanity.com/paper/2605.30900/) (2026)
- [Event-based Batting Impact Estimation](https://mlsanity.com/paper/2605.25656/) (2026)
- [1000 Rallies: An Event-Camera Dataset and Real-Time Learned Ball-State Estimation for Robotic Table Tennis](https://mlsanity.com/paper/2606.25620/) (2026)

## Abstract (arXiv)

In this paper, we present a real-time egocentric trajectory prediction system for table tennis using event cameras. Unlike standard cameras, which suffer from high latency and motion blur at fast ball speeds, event cameras provide higher temporal resolution, allowing more frequent state updates, greater robustness to outliers, and accurate trajectory predictions using just a short time window after the opponent's impact. We collect a dataset of ping-pong game sequences, including 3D ground-truth trajectories of the ball, synchronized with sensor data from the Meta Project Aria glasses and event streams. Our system leverages foveated vision, using eye-gaze data from the glasses to process only events in the viewer's fovea. This biologically inspired approach improves ball detection performance and significantly reduces computational latency, as it efficiently allocates resources to the most perceptually relevant regions, achieving a reduction factor of 10.81 on the collected trajectories. Our detection pipeline has a worst-case total latency of 4.5 ms, including computation and perception - significantly lower than a frame-based 30 FPS system, which, in the worst case, takes 66 ms solely for perception. Finally, we fit a trajectory prediction model to the estimated states of the ball, enabling 3D trajectory forecasting in the future. To the best of our knowledge, this is the first approach to predict table tennis trajectories from an egocentric perspective using event cameras.