# Motion Capture from Pan-Tilt Cameras with Unknown Orientation

arXiv: https://arxiv.org/abs/1908.11676 · HTML: https://mlsanity.com/paper/1908.11676/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Aug 2019.

## Built on by (most cited first, 6 of 6)

- [FLEX: Extrinsic Parameters-free Multi-view 3D Human Motion Reconstruction](https://mlsanity.com/paper/2105.01937/) (2021)
- [SmartMocap: Joint Estimation of Human and Camera Motion using Uncalibrated RGB Cameras](https://mlsanity.com/paper/2209.13906/) (2022)
- [Tracking Skiers from the Top to the Bottom](https://mlsanity.com/paper/2312.09723/) (2023)
- [Improved 2D Keypoint Detection in Out-of-Balance and Fall Situations -- combining input rotations and a kinematic model](https://mlsanity.com/paper/2112.12193/) (2021)
- [Visualizing Skiers' Trajectories in Monocular Videos](https://mlsanity.com/paper/2304.02994/) (2023)
- [Video-Based Reconstruction of the Trajectories Performed by Skiers](https://mlsanity.com/paper/2112.09647/) (2021)

## Abstract (arXiv)

In sports, such as alpine skiing, coaches would like to know the speed and various biomechanical variables of their athletes and competitors. Existing methods use either body-worn sensors, which are cumbersome to setup, or manual image annotation, which is time consuming. We propose a method for estimating an athlete's global 3D position and articulated pose using multiple cameras. By contrast to classical markerless motion capture solutions, we allow cameras to rotate freely so that large capture volumes can be covered. In a first step, tight crops around the skier are predicted and fed to a 2D pose estimator network. The 3D pose is then reconstructed using a bundle adjustment method. Key to our solution is the rotation estimation of Pan-Tilt cameras in a joint optimization with the athlete pose and conditioning on relative background motion computed with feature tracking. Furthermore, we created a new alpine skiing dataset and annotated it with 2D pose labels, to overcome shortcomings of existing ones. Our method estimates accurate global 3D poses from images only and provides coaches with an automatic and fast tool for measuring and improving an athlete's performance.