# D3PRefiner: A Diffusion-based Denoise Method for 3D Human Pose Refinement

arXiv: https://arxiv.org/abs/2401.03914 · HTML: https://mlsanity.com/paper/2401.03914/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 48 references that are in the map.
- First posted on arXiv 8 Jan 2024.

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [3D human pose estimation in video with temporal convolutions and semi-supervised training](https://mlsanity.com/paper/1811.11742/) (2018)
- [A simple yet effective baseline for 3d human pose estimation](https://mlsanity.com/paper/1705.03098/) (2017)
- [Cascaded Pyramid Network for Multi-Person Pose Estimation](https://mlsanity.com/paper/1711.07319/) (2017)
- [Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Pose](https://mlsanity.com/paper/1611.07828/) (2016)
- [MixSTE: Seq2seq Mixed Spatio-Temporal Encoder for 3D Human Pose Estimation in Video](https://mlsanity.com/paper/2203.00859/) (2022)
- [Ordinal Depth Supervision for 3D Human Pose Estimation](https://mlsanity.com/paper/1805.04095/) (2018)
- [PointCleanNet: Learning to Denoise and Remove Outliers from Dense Point Clouds](https://mlsanity.com/paper/1901.01060/) (2019)
- [PoseFormerV2: Exploring Frequency Domain for Efficient and Robust 3D Human Pose Estimation](https://mlsanity.com/paper/2303.17472/) (2023)
- [Exploiting temporal information for 3D pose estimation](https://mlsanity.com/paper/1711.08585/) (2017)
- [Motion Guided 3D Pose Estimation from Videos](https://mlsanity.com/paper/2004.13985/) (2020)
- [Robotic Telekinesis: Learning a Robotic Hand Imitator by Watching Humans on Youtube](https://mlsanity.com/paper/2202.10448/) (2022)
- [Diffusion-Based 3D Human Pose Estimation with Multi-Hypothesis Aggregation](https://mlsanity.com/paper/2303.11579/) (2023)
- [P-STMO: Pre-Trained Spatial Temporal Many-to-One Model for 3D Human Pose Estimation](https://mlsanity.com/paper/2203.07628/) (2022)
- [SmoothNet: A Plug-and-Play Network for Refining Human Poses in Videos](https://mlsanity.com/paper/2112.13715/) (2021)
- [Conditional Directed Graph Convolution for 3D Human Pose Estimation](https://mlsanity.com/paper/2107.07797/) (2021)
- [PoseFix: Model-agnostic General Human Pose Refinement Network](https://mlsanity.com/paper/1812.03595/) (2018)
- [Graph-PCNN: Two Stage Human Pose Estimation with Graph Pose Refinement](https://mlsanity.com/paper/2007.10599/) (2020)
- [DiffuPose: Monocular 3D Human Pose Estimation via Denoising Diffusion Probabilistic Model](https://mlsanity.com/paper/2212.02796/) (2022)
- [Recurrent 3D Pose Sequence Machines](https://mlsanity.com/paper/1707.09695/) (2017)
- [3D human pose estimation from depth maps using a deep combination of poses](https://mlsanity.com/paper/1807.05389/) (2018)
- [Deep Active Visual Attention for Real-time Robot Motion Generation: Emergence of Tool-body Assimilation and Adaptive Tool-use](https://mlsanity.com/paper/2206.14530/) (2022)

## Abstract (arXiv)

Three-dimensional (3D) human pose estimation using a monocular camera has gained increasing attention due to its ease of implementation and the abundance of data available from daily life. However, owing to the inherent depth ambiguity in images, the accuracy of existing monocular camera-based 3D pose estimation methods remains unsatisfactory, and the estimated 3D poses usually include much noise. By observing the histogram of this noise, we find each dimension of the noise follows a certain distribution, which indicates the possibility for a neural network to learn the mapping between noisy poses and ground truth poses. In this work, in order to obtain more accurate 3D poses, a Diffusion-based 3D Pose Refiner (D3PRefiner) is proposed to refine the output of any existing 3D pose estimator. We first introduce a conditional multivariate Gaussian distribution to model the distribution of noisy 3D poses, using paired 2D poses and noisy 3D poses as conditions to achieve greater accuracy. Additionally, we leverage the architecture of current diffusion models to convert the distribution of noisy 3D poses into ground truth 3D poses. To evaluate the effectiveness of the proposed method, two state-of-the-art sequence-to-sequence 3D pose estimators are used as basic 3D pose estimation models, and the proposed method is evaluated on different types of 2D poses and different lengths of the input sequence. Experimental results demonstrate the proposed architecture can significantly improve the performance of current sequence-to-sequence 3D pose estimators, with a reduction of at least 10.3% in the mean per joint position error (MPJPE) and at least 11.0% in the Procrustes MPJPE (P-MPJPE).