# ProtoRes: Proto-Residual Network for Pose Authoring via Learned Inverse Kinematics

arXiv: https://arxiv.org/abs/2106.01981 · HTML: https://mlsanity.com/paper/2106.01981/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 52 references that are in the map.
- First posted on arXiv 3 Jun 2021.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [Structural-RNN: Deep Learning on Spatio-Temporal Graphs](https://mlsanity.com/paper/1511.05298/) (2015)
- [Robust Motion In-betweening](https://mlsanity.com/paper/2102.04942/) (2021)
- [QuaterNet: A Quaternion-based Recurrent Model for Human Motion](https://mlsanity.com/paper/1805.06485/) (2018)
- [LoBSTr: Real-time Lower-body Pose Prediction from Sparse Upper-body Tracking Signals](https://mlsanity.com/paper/2103.01500/) (2021)
- [Real-time Deep Pose Estimation with Geodesic Loss for Image-to-Template Rigid Registration](https://mlsanity.com/paper/1803.05982/) (2018)
- [Space-Time Representation of People Based on 3D Skeletal Data: A Review](https://mlsanity.com/paper/1601.01006/) (2016)
- [Graph and Temporal Convolutional Networks for 3D Multi-person Pose Estimation in Monocular Videos](https://mlsanity.com/paper/2012.11806/) (2020)
- [FixMyPose: Pose Correctional Captioning and Retrieval](https://mlsanity.com/paper/2104.01703/) (2021)
- [Classifying and Visualizing Motion Capture Sequences using Deep Neural Networks](https://mlsanity.com/paper/1306.3874/) (2013)
- [Metric Pose Estimation for Human-Machine Interaction Using Monocular Vision](https://mlsanity.com/paper/1910.03239/) (2019)
- [AnimGAN: A Spatiotemporally-Conditioned Generative Adversarial Network for Character Animation](https://mlsanity.com/paper/2005.11489/) (2020)

## Built on by (most cited first, 17 of 17)

- [BoDiffusion: Diffusing Sparse Observations for Full-Body Human Motion Synthesis](https://mlsanity.com/paper/2304.11118/) (2023)
- [NIKI: Neural Inverse Kinematics with Invertible Neural Networks for 3D Human Pose and Shape Estimation](https://mlsanity.com/paper/2305.08590/) (2023)
- [HanDiffuser: Text-to-Image Generation With Realistic Hand Appearances](https://mlsanity.com/paper/2403.01693/) (2024)
- [PoseFix: Correcting 3D Human Poses with Natural Language](https://mlsanity.com/paper/2309.08480/) (2023)
- [Iterative Motion Editing with Natural Language](https://mlsanity.com/paper/2312.11538/) (2023)
- [SimMotionEdit: Text-Based Human Motion Editing with Motion Similarity Prediction](https://mlsanity.com/paper/2503.18211/) (2025)
- [Pretrained Diffusion Models for Unified Human Motion Synthesis](https://mlsanity.com/paper/2212.02837/) (2022)
- [Normalizing Flows on the Product Space of SO(3) Manifolds for Probabilistic Human Pose Modeling](https://mlsanity.com/paper/2404.05675/) (2024)
- [Inverse Kinematics for Neuro-Robotic Grasping with Humanoid Embodied Agents](https://mlsanity.com/paper/2404.08825/) (2024)
- [SMPL-IK: Learned Morphology-Aware Inverse Kinematics for AI Driven Artistic Workflows](https://mlsanity.com/paper/2208.08274/) (2022)
- [CycleIK: Neuro-inspired Inverse Kinematics](https://mlsanity.com/paper/2307.11554/) (2023)
- [Omni-Supervised Motion Editing: Balancing Change and Invariance through Positive-Negative Learning](https://mlsanity.com/paper/2605.30969/) (2026)
- [DragPoser: Motion Reconstruction from Variable Sparse Tracking Signals via Latent Space Optimization](https://mlsanity.com/paper/2406.14567/) (2024)
- [Learnable SMPLify: A Neural Solution for Optimization-Free Human Pose Inverse Kinematics](https://mlsanity.com/paper/2508.13562/) (2025)
- [MODL: Multilearner Online Deep Learning](https://mlsanity.com/paper/2405.18281/) (2024)
- [Spatial Temporal Synergy: Balancing Change and Invariance in Text Driven 3D Human Motion Editing](https://mlsanity.com/paper/2608.16008/) (2026)
- [Probabilistic Pretraining for Neural Regression](https://mlsanity.com/paper/2508.16355/) (2025)

## Abstract (arXiv)

Our work focuses on the development of a learnable neural representation of human pose for advanced AI assisted animation tooling. Specifically, we tackle the problem of constructing a full static human pose based on sparse and variable user inputs (e.g. locations and/or orientations of a subset of body joints). To solve this problem, we propose a novel neural architecture that combines residual connections with prototype encoding of a partially specified pose to create a new complete pose from the learned latent space. We show that our architecture outperforms a baseline based on Transformer, both in terms of accuracy and computational efficiency. Additionally, we develop a user interface to integrate our neural model in Unity, a real-time 3D development platform. Furthermore, we introduce two new datasets representing the static human pose modeling problem, based on high-quality human motion capture data, which will be released publicly along with model code.