# Towards Single Camera Human 3D-Kinematics

arXiv: https://arxiv.org/abs/2301.05435 · HTML: https://mlsanity.com/paper/2301.05435/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 66 references that are in the map.
- First posted on arXiv 13 Jan 2023.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors](https://mlsanity.com/paper/2207.02696/) (2022)
- [3D human pose estimation in video with temporal convolutions and semi-supervised training](https://mlsanity.com/paper/1811.11742/) (2018)
- [RMPE: Regional Multi-person Pose Estimation](https://mlsanity.com/paper/1612.00137/) (2016)
- [Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Pose](https://mlsanity.com/paper/1611.07828/) (2016)
- [Markerless tracking of user-defined features with deep learning](https://mlsanity.com/paper/1804.03142/) (2018)
- [VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera](https://mlsanity.com/paper/1705.01583/) (2017)
- [BlazePose: On-device Real-time Body Pose tracking](https://mlsanity.com/paper/2006.10204/) (2020)
- [Dive into Deep Learning](https://mlsanity.com/paper/2106.11342/) (2021)
- [MoVi: A Large Multipurpose Motion and Video Dataset](https://mlsanity.com/paper/2003.01888/) (2020)
- [3D Human Pose Estimation using Spatio-Temporal Networks with Explicit Occlusion Training](https://mlsanity.com/paper/2004.11822/) (2020)
- [3D Human Pose Estimation with 2D Marginal Heatmaps](https://mlsanity.com/paper/1806.01484/) (2018)
- [Movement science needs different pose tracking algorithms](https://mlsanity.com/paper/1907.10226/) (2019)
- [Metric-Scale Truncation-Robust Heatmaps for 3D Human Pose Estimation](https://mlsanity.com/paper/2003.02953/) (2020)
- [Enhanced 3D Human Pose Estimation from Videos by using Attention-Based Neural Network with Dilated Convolutions](https://mlsanity.com/paper/2103.03170/) (2021)

## Built on by (most cited first, 8 of 8)

- [From Skin to Skeleton: Towards Biomechanically Accurate 3D Digital Humans](https://mlsanity.com/paper/2509.06607/) (2025)
- [Reconstructing Humans with a Biomechanically Accurate Skeleton](https://mlsanity.com/paper/2503.21751/) (2025)
- [BioPose: Biomechanically-accurate 3D Pose Estimation from Monocular Videos](https://mlsanity.com/paper/2501.07800/) (2025)
- [OpenCapBench: A Benchmark to Bridge Pose Estimation and Biomechanics](https://mlsanity.com/paper/2406.09788/) (2024)
- [3D Kinematics Estimation from Video with a Biomechanical Model and Synthetic Training Data](https://mlsanity.com/paper/2402.13172/) (2024)
- [MonoMSK: Monocular 3D Musculoskeletal Dynamics Estimation](https://mlsanity.com/paper/2511.19326/) (2025)
- [TrackStudio: An Integrated Toolkit for Markerless Tracking](https://mlsanity.com/paper/2511.07624/) (2025)
- [BioHuman: Learning Biomechanical Human Representations from Video](https://mlsanity.com/paper/2605.14772/) (2026)

## Abstract (arXiv)

Markerless estimation of 3D Kinematics has the great potential to clinically diagnose and monitor movement disorders without referrals to expensive motion capture labs; however, current approaches are limited by performing multiple de-coupled steps to estimate the kinematics of a person from videos. Most current techniques work in a multi-step approach by first detecting the pose of the body and then fitting a musculoskeletal model to the data for accurate kinematic estimation. Errors in training data of the pose detection algorithms, model scaling, as well the requirement of multiple cameras limit the use of these techniques in a clinical setting. Our goal is to pave the way toward fast, easily applicable and accurate 3D kinematic estimation xdeletedin a clinical setting. To this end, we propose a novel approach for direct 3D human kinematic estimation D3KE from videos using deep neural networks. Our experiments demonstrate that the proposed end-to-end training is robust and outperforms 2D and 3D markerless motion capture based kinematic estimation pipelines in terms of joint angles error by a large margin (35% from 5.44 to 3.54 degrees). We show that D3KE is superior to the multi-step approach and can run at video framerate speeds. This technology shows the potential for clinical analysis from mobile devices in the future.