# Towards Learning Monocular 3D Object Localization From 2D Labels using the Physical Laws of Motion

arXiv: https://arxiv.org/abs/2310.17462 · HTML: https://mlsanity.com/paper/2310.17462/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 32 references that are in the map.
- First posted on arXiv 26 Oct 2023.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Hamiltonian Neural Networks](https://mlsanity.com/paper/1906.01563/) (2019)
- [ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders](https://mlsanity.com/paper/2301.00808/) (2023)
- [Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulation](https://mlsanity.com/paper/2106.13281/) (2021)
- [DeepSportradar-v1: Computer Vision Dataset for Sports Understanding with High Quality Annotations](https://mlsanity.com/paper/2208.08190/) (2022)
- [MonoTrack: Shuttle trajectory reconstruction from monocular badminton video](https://mlsanity.com/paper/2204.01899/) (2022)
- [Table Tennis Stroke Recognition Using Two-Dimensional Human Pose Estimation](https://mlsanity.com/paper/2104.09907/) (2021)
- [What Players do with the Ball: A Physically Constrained Interaction Modeling](https://mlsanity.com/paper/1511.06181/) (2015)
- [Ball 3D Localization From A Single Calibrated Image](https://mlsanity.com/paper/2204.00003/) (2022)
- [Activity-conditioned continuous human pose estimation for performance analysis of athletes using the example of swimming](https://mlsanity.com/paper/1802.00634/) (2018)
- [Recognition of Freely Selected Keypoints on Human Limbs](https://mlsanity.com/paper/2204.06326/) (2022)

## Built on by (most cited first, 3 of 3)

- [Towards Ball Spin and Trajectory Analysis in Table Tennis Broadcast Videos via Physically Grounded Synthetic-to-Real Transfer](https://mlsanity.com/paper/2504.19863/) (2025)
- [Uplifting Table Tennis: A Robust, Real-World Application for 3D Trajectory and Spin Estimation](https://mlsanity.com/paper/2511.20250/) (2025)
- [Physics-Informed Tracking (PIT)](https://mlsanity.com/paper/2604.16895/) (2026)

## Abstract (arXiv)

We present a novel method for precise 3D object localization in single images from a single calibrated camera using only 2D labels. No expensive 3D labels are needed. Thus, instead of using 3D labels, our model is trained with easy-to-annotate 2D labels along with the physical knowledge of the object's motion. Given this information, the model can infer the latent third dimension, even though it has never seen this information during training. Our method is evaluated on both synthetic and real-world datasets, and we are able to achieve a mean distance error of just 6 cm in our experiments on real data. The results indicate the method's potential as a step towards learning 3D object location estimation, where collecting 3D data for training is not feasible.