# Uplifting Table Tennis: A Robust, Real-World Application for 3D Trajectory and Spin Estimation

arXiv: https://arxiv.org/abs/2511.20250 · HTML: https://mlsanity.com/paper/2511.20250/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 44 references that are in the map.
- First posted on arXiv 25 Nov 2025.

## Builds on (most cited first, 28 of 28)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [Token Merging: Your ViT But Faster](https://mlsanity.com/paper/2210.09461/) (2022)
- [ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation](https://mlsanity.com/paper/2204.12484/) (2022)
- [Convolutional Pose Machines](https://mlsanity.com/paper/1602.00134/) (2016)
- [TrackNet: A Deep Learning Network for Tracking High-speed and Tiny Objects in Sports Applications](https://mlsanity.com/paper/1907.03698/) (2019)
- [MonoTrack: Shuttle trajectory reconstruction from monocular badminton video](https://mlsanity.com/paper/2204.01899/) (2022)
- [What Players do with the Ball: A Physically Constrained Interaction Modeling](https://mlsanity.com/paper/1511.06181/) (2015)
- [Ball 3D Localization From A Single Calibrated Image](https://mlsanity.com/paper/2204.00003/) (2022)
- [Table tennis ball spin estimation with an event camera](https://mlsanity.com/paper/2404.09870/) (2024)
- [Widely Applicable Strong Baseline for Sports Ball Detection and Tracking](https://mlsanity.com/paper/2311.05237/) (2023)
- [Segformer++: Efficient Token-Merging Strategies for High-Resolution Semantic Segmentation](https://mlsanity.com/paper/2405.14467/) (2024)
- [SpinDOE: A ball spin estimation method for table tennis robot](https://mlsanity.com/paper/2303.03879/) (2023)
- [Recognition of Freely Selected Keypoints on Human Limbs](https://mlsanity.com/paper/2204.06326/) (2022)
- [LATTE-MV: Learning to Anticipate Table Tennis Hits from Monocular Videos](https://mlsanity.com/paper/2503.20936/) (2025)
- [TT3D: Table Tennis 3D Reconstruction](https://mlsanity.com/paper/2504.10035/) (2025)
- [Multi-Camera Asynchronous Ball Localization and Trajectory Prediction with Factor Graphs and Human Poses](https://mlsanity.com/paper/2401.17185/) (2024)
- [Towards Ball Spin and Trajectory Analysis in Table Tennis Broadcast Videos via Physically Grounded Synthetic-to-Real Transfer](https://mlsanity.com/paper/2504.19863/) (2025)
- [KaliCalib: A Framework for Basketball Court Registration](https://mlsanity.com/paper/2209.07795/) (2022)
- [Towards Learning Monocular 3D Object Localization From 2D Labels using the Physical Laws of Motion](https://mlsanity.com/paper/2310.17462/) (2023)
- [Rink-Agnostic Hockey Rink Registration](https://mlsanity.com/paper/2401.01003/) (2023)
- [Context-Aware 3D Object Localization from Single Calibrated Images: A Study of Basketballs](https://mlsanity.com/paper/2309.03640/) (2023)
- [Where Is The Ball: 3D Ball Trajectory Estimation From 2D Monocular Tracking](https://mlsanity.com/paper/2506.05763/) (2025)
- [BlurBall: Joint Ball and Motion Blur Estimation for Table Tennis Ball Tracking](https://mlsanity.com/paper/2509.18387/) (2025)

## Built on by (most cited first, 3 of 3)

- [Stepping VLMs onto the Court: Benchmarking Spatial Intelligence in Sports](https://mlsanity.com/paper/2603.09896/) (2026)
- [TT4D: A Pipeline and Dataset for Table Tennis 4D Reconstruction From Monocular Videos](https://mlsanity.com/paper/2605.01234/) (2026)
- [BilliardPhys-Bench: Benchmarking Physical Reasoning and Visual Dynamics of Multimodal LLMs](https://mlsanity.com/paper/2605.30900/) (2026)

## Abstract (arXiv)

Obtaining the precise 3D motion of a table tennis ball from standard monocular videos is a challenging problem, as existing methods trained on synthetic data struggle to generalize to the noisy, imperfect ball and table detections of the real world. This is primarily due to the inherent lack of 3D ground truth trajectories and spin annotations for real-world video. To overcome this, we propose a novel two-stage pipeline that divides the problem into a front-end perception task and a back-end 2D-to-3D uplifting task. This separation allows us to train the front-end components with abundant 2D supervision from our newly created TTHQ dataset, while the back-end uplifting network is trained exclusively on physically-correct synthetic data. We specifically re-engineer the uplifting model to be robust to common real-world artifacts, such as missing detections and varying frame rates. By integrating a ball detector and a table keypoint detector, our approach transforms a proof-of-concept uplifting method into a practical, robust, and high-performing end-to-end application for 3D table tennis trajectory and spin analysis.