# Where Is The Ball: 3D Ball Trajectory Estimation From 2D Monocular Tracking

arXiv: https://arxiv.org/abs/2506.05763 · HTML: https://mlsanity.com/paper/2506.05763/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 58 references that are in the map.
- First posted on arXiv 6 Jun 2025.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [AdaBins: Depth Estimation using Adaptive Bins](https://mlsanity.com/paper/2011.14141/) (2020)
- [Generating Sequences With Recurrent Neural Networks](https://mlsanity.com/paper/1308.0850/) (2013)
- [Contact and Human Dynamics from Monocular Video](https://mlsanity.com/paper/2007.11678/) (2020)
- [Improved Neural Relation Detection for Knowledge Base Question Answering](https://mlsanity.com/paper/1704.06194/) (2017)
- [Newtonian Image Understanding: Unfolding the Dynamics of Objects in Static Images](https://mlsanity.com/paper/1511.04048/) (2015)
- [Soccer on Your Tabletop](https://mlsanity.com/paper/1806.00890/) (2018)
- [Motion-from-Blur: 3D Shape and Motion Estimation of Motion-blurred Objects in Videos](https://mlsanity.com/paper/2111.14465/) (2021)
- [Shape from Blur: Recovering Textured 3D Shape and Motion of Fast Moving Objects](https://mlsanity.com/paper/2106.08762/) (2021)
- [Bounce and Learn: Modeling Scene Dynamics with Real-World Bounces](https://mlsanity.com/paper/1904.06827/) (2019)
- [Neural Re-Simulation for Generating Bounces in Single Images](https://mlsanity.com/paper/1908.06217/) (2019)
- [Estimating 3D Trajectories from 2D Projections via Disjunctive Factored Four-Way Conditional Restricted Boltzmann Machines](https://mlsanity.com/paper/1604.05865/) (2016)
- [Weighted Contrastive Divergence](https://mlsanity.com/paper/1801.02567/) (2018)
- [SMASH: Physics-guided Reconstruction of Collisions from Videos](https://mlsanity.com/paper/1603.08984/) (2016)

## Built on by (most cited first, 2 of 2)

- [Uplifting Table Tennis: A Robust, Real-World Application for 3D Trajectory and Spin Estimation](https://mlsanity.com/paper/2511.20250/) (2025)
- [TT4D: A Pipeline and Dataset for Table Tennis 4D Reconstruction From Monocular Videos](https://mlsanity.com/paper/2605.01234/) (2026)

## Abstract (arXiv)

We present a method for 3D ball trajectory estimation from a 2D tracking sequence. To overcome the ambiguity in 3D from 2D estimation, we design an LSTM-based pipeline that utilizes a novel canonical 3D representation that is independent of the camera's location to handle arbitrary views and a series of intermediate representations that encourage crucial invariance and reprojection consistency. We evaluated our method on four synthetic and three real datasets and conducted extensive ablation studies on our design choices. Despite training solely on simulated data, our method achieves state-of-the-art performance and can generalize to real-world scenarios with multiple trajectories, opening up a range of applications in sport analysis and virtual replay. Please visit our page: https://where-is-the-ball.github.io.