# 1000 Rallies: An Event-Camera Dataset and Real-Time Learned Ball-State Estimation for Robotic Table Tennis

arXiv: https://arxiv.org/abs/2606.25620 · HTML: https://mlsanity.com/paper/2606.25620/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 36 references that are in the map.
- First posted on arXiv 24 Jun 2026.

## Builds on (most cited first, 21 of 21)

- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [Learning to Detect Objects with a 1 Megapixel Event Camera](https://mlsanity.com/paper/2009.13436/) (2020)
- [Event Stream-based Visual Object Tracking: A High-Resolution Benchmark Dataset and A Novel Baseline](https://mlsanity.com/paper/2309.14611/) (2023)
- [How to Calibrate Your Event Camera](https://mlsanity.com/paper/2105.12362/) (2021)
- [TTNet: Real-time temporal and spatial video analysis of table tennis](https://mlsanity.com/paper/2004.09927/) (2020)
- [HITTER: A HumanoId Table TEnnis Robot via Hierarchical Planning and Learning](https://mlsanity.com/paper/2508.21043/) (2025)
- [Achieving Human Level Competitive Robot Table Tennis](https://mlsanity.com/paper/2408.03906/) (2024)
- [Event-based Agile Object Catching with a Quadrupedal Robot](https://mlsanity.com/paper/2303.17479/) (2023)
- [Learning coordinated badminton skills for legged manipulators](https://mlsanity.com/paper/2505.22974/) (2025)
- [Jerk-limited Real-time Trajectory Generation with Arbitrary Target States](https://mlsanity.com/paper/2105.04830/) (2021)
- [EV-Catcher: High-Speed Object Catching Using Low-latency Event-based Neural Networks](https://mlsanity.com/paper/2304.07200/) (2023)
- [Real Time Trajectory Prediction Using Deep Conditional Generative Models](https://mlsanity.com/paper/1909.03895/) (2019)
- [Spin Detection in Robotic Table Tennis](https://mlsanity.com/paper/1905.07967/) (2019)
- [SpinDOE: A ball spin estimation method for table tennis robot](https://mlsanity.com/paper/2303.03879/) (2023)
- [SpikeMOT: Event-based Multi-Object Tracking with Sparse Motion Features](https://mlsanity.com/paper/2309.16987/) (2023)
- [An Event-Based Perception Pipeline for a Table Tennis Robot](https://mlsanity.com/paper/2502.00749/) (2025)
- [Detection of Fast-Moving Objects with Neuromorphic Hardware](https://mlsanity.com/paper/2403.10677/) (2024)
- [Integrating Learning-Based Manipulation and Physics-Based Locomotion for Whole-Body Badminton Robot Control](https://mlsanity.com/paper/2504.17771/) (2025)
- [Egocentric Event-Based Vision for Ping Pong Ball Trajectory Prediction](https://mlsanity.com/paper/2506.07860/) (2025)
- [Event-based Visual Tracking in Dynamic Environments](https://mlsanity.com/paper/2212.07754/) (2022)

## Abstract (arXiv)

Robotic table tennis has emerged as a compelling benchmark for real-time robotic perception due to its fast ball dynamics and stringent timing requirements. Accurate, high-frequency, and low-latency ball state estimation is critical for reliable trajectory prediction and timely control. Traditional frame-based cameras face an inherent trade-off: low frame rates leave temporal blind spots that miss fast-moving objects and high frame rates raise data and computational cost. Event cameras instead offer microsecond temporal resolution and, under sufficient illumination, remain largely free of motion blur even at high ball speeds. However, the community lacks large-scale datasets to develop and benchmark event-based perception in realistic sports scenarios. We address this gap by introducing the first large-scale event-camera dataset for table tennis, comprising over 1000 rallies from a diverse group of players ranging from amateurs to elite-level athletes. Each recording captures the event stream alongside 14 synchronized high-speed frame-based cameras at 200 FPS, which we use to produce 1 kHz pseudo ground-truth labels for ball position, velocity, and spin. Building on this dataset, we train a convolutional neural network robust to background player motion that jointly estimates the ball's position and velocity in the image-plane from events. Treating the predicted velocity as an additional measurement in the Kalman filter reduces bounce-point prediction error by 36% relative to a position-only baseline. Finally, we close the perception-action loop by integrating the event-based system with a St"aubli robotic arm, enabling the first real-time human-robot table tennis rallies driven by event-based perception.