# Towards Ball Spin and Trajectory Analysis in Table Tennis Broadcast Videos via Physically Grounded Synthetic-to-Real Transfer

arXiv: https://arxiv.org/abs/2504.19863 · HTML: https://mlsanity.com/paper/2504.19863/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 47 references that are in the map.
- First posted on arXiv 28 Apr 2025.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [3D Human Pose Estimation with Spatial and Temporal Transformers](https://mlsanity.com/paper/2103.10455/) (2021)
- [ViTPose++: Vision Transformer for Generic Body Pose Estimation](https://mlsanity.com/paper/2212.04246/) (2022)
- [Uplift and Upsample: Efficient 3D Human Pose Estimation with Uplifting Transformers](https://mlsanity.com/paper/2210.06110/) (2022)
- [Achieving Human Level Competitive Robot Table Tennis](https://mlsanity.com/paper/2408.03906/) (2024)
- [MonoTrack: Shuttle trajectory reconstruction from monocular badminton video](https://mlsanity.com/paper/2204.01899/) (2022)
- [What Players do with the Ball: A Physically Constrained Interaction Modeling](https://mlsanity.com/paper/1511.06181/) (2015)
- [Ball 3D Localization From A Single Calibrated Image](https://mlsanity.com/paper/2204.00003/) (2022)
- [Monocular 3D Human Pose Estimation for Sports Broadcasts using Partial Sports Field Registration](https://mlsanity.com/paper/2304.04437/) (2023)
- [Table tennis ball spin estimation with an event camera](https://mlsanity.com/paper/2404.09870/) (2024)
- [Segformer++: Efficient Token-Merging Strategies for High-Resolution Semantic Segmentation](https://mlsanity.com/paper/2405.14467/) (2024)
- [SpinDOE: A ball spin estimation method for table tennis robot](https://mlsanity.com/paper/2303.03879/) (2023)
- [Recognition of Freely Selected Keypoints on Human Limbs](https://mlsanity.com/paper/2204.06326/) (2022)
- [Black-Box vs. Gray-Box: A Case Study on Learning Table Tennis Ball Trajectory Prediction with Spin and Impacts](https://mlsanity.com/paper/2305.15189/) (2023)
- [Multi-Camera Asynchronous Ball Localization and Trajectory Prediction with Factor Graphs and Human Poses](https://mlsanity.com/paper/2401.17185/) (2024)
- [Decoupling Video and Human Motion: Towards Practical Event Detection in Athlete Recordings](https://mlsanity.com/paper/2004.09776/) (2020)
- [Towards Learning Monocular 3D Object Localization From 2D Labels using the Physical Laws of Motion](https://mlsanity.com/paper/2310.17462/) (2023)
- [Context-Aware 3D Object Localization from Single Calibrated Images: A Study of Basketballs](https://mlsanity.com/paper/2309.03640/) (2023)
- [Learning Dynamics of a Ball with Differentiable Factor Graph and Roto-Translational Invariant Representations](https://mlsanity.com/paper/2409.16467/) (2024)

## Built on by (most cited first, 4 of 4)

- [Uplifting Table Tennis: A Robust, Real-World Application for 3D Trajectory and Spin Estimation](https://mlsanity.com/paper/2511.20250/) (2025)
- [RacketVision: A Multiple Racket Sports Benchmark for Unified Ball and Racket Analysis](https://mlsanity.com/paper/2511.17045/) (2025)
- [BlurBall: Joint Ball and Motion Blur Estimation for Table Tennis Ball Tracking](https://mlsanity.com/paper/2509.18387/) (2025)
- [Estimating Velocity and Spin of Spherical Objects from Rolling-Shutter Image(s)](https://mlsanity.com/paper/2606.31760/) (2026)

## Abstract (arXiv)

Analyzing a player's technique in table tennis requires knowledge of the ball's 3D trajectory and spin. While, the spin is not directly observable in standard broadcasting videos, we show that it can be inferred from the ball's trajectory in the video. We present a novel method to infer the initial spin and 3D trajectory from the corresponding 2D trajectory in a video. Without ground truth labels for broadcast videos, we train a neural network solely on synthetic data. Due to the choice of our input data representation, physically correct synthetic training data, and using targeted augmentations, the network naturally generalizes to real data. Notably, these simple techniques are sufficient to achieve generalization. No real data at all is required for training. To the best of our knowledge, we are the first to present a method for spin and trajectory prediction in simple monocular broadcast videos, achieving an accuracy of 92.0% in spin classification and a 2D reprojection error of 0.19% of the image diagonal.