# 3D-Aware Neural Body Fitting for Occlusion Robust 3D Human Pose Estimation

arXiv: https://arxiv.org/abs/2308.10123 · HTML: https://mlsanity.com/paper/2308.10123/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 63 of its 85 references that are in the map.
- First posted on arXiv 19 Aug 2023.

## Builds on (most cited first, 30 of 63)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [End-to-end Recovery of Human Shape and Pose](https://mlsanity.com/paper/1712.06584/) (2017)
- [PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization](https://mlsanity.com/paper/1905.05172/) (2019)
- [GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields](https://mlsanity.com/paper/2011.12100/) (2020)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image](https://mlsanity.com/paper/1607.08128/) (2016)
- [Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans](https://mlsanity.com/paper/2012.15838/) (2020)
- [Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loop](https://mlsanity.com/paper/1909.12828/) (2019)
- [Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervision](https://mlsanity.com/paper/1611.09813/) (2016)
- [HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Video](https://mlsanity.com/paper/2201.04127/) (2022)
- [Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoning](https://mlsanity.com/paper/1904.01786/) (2019)
- [PARE: Part Attention Regressor for 3D Human Body Estimation](https://mlsanity.com/paper/2104.08527/) (2021)
- [ICON: Implicit Clothed humans Obtained from Normals](https://mlsanity.com/paper/2112.09127/) (2021)
- [Mesh Graphormer](https://mlsanity.com/paper/2104.00272/) (2021)
- [Integral Human Pose Regression](https://mlsanity.com/paper/1711.08229/) (2017)
- [NeuMan: Neural Human Radiance Field from a Single Video](https://mlsanity.com/paper/2203.12575/) (2022)
- [Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies](https://mlsanity.com/paper/1801.01615/) (2018)
- [A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Pose](https://mlsanity.com/paper/2102.06199/) (2021)
- [Learning to Estimate 3D Human Pose and Shape from a Single Color Image](https://mlsanity.com/paper/1805.04092/) (2018)
- [Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Pose](https://mlsanity.com/paper/1611.07828/) (2016)
- [Unite the People: Closing the Loop Between 3D and 2D Human Representations](https://mlsanity.com/paper/1701.02468/) (2017)
- [Neural Articulated Radiance Field](https://mlsanity.com/paper/2104.03110/) (2021)

## Built on by (most cited first, 28 of 28)

- [pix2gestalt: Amodal Segmentation by Synthesizing Wholes](https://mlsanity.com/paper/2401.14398/) (2024)
- [RoHM: Robust Human Motion Reconstruction via Diffusion](https://mlsanity.com/paper/2401.08570/) (2024)
- [PhysPT: Physics-aware Pretrained Transformer for Estimating Human Dynamics from Monocular Videos](https://mlsanity.com/paper/2404.04430/) (2024)
- [Learning Occlusion-Robust Vision Transformers for Real-Time UAV Tracking](https://mlsanity.com/paper/2504.09228/) (2025)
- [CoMotion: Concurrent Multi-person 3D Motion](https://mlsanity.com/paper/2504.12186/) (2025)
- [Application of Multimodal Fusion Deep Learning Model in Disease Recognition](https://mlsanity.com/paper/2406.18546/) (2024)
- [Exploration of Attention Mechanism-Enhanced Deep Learning Models in the Mining of Medical Textual Data](https://mlsanity.com/paper/2406.00016/) (2024)
- [Enhancing Medical Imaging with GANs Synthesizing Realistic Images from Limited Data](https://mlsanity.com/paper/2406.18547/) (2024)
- [Research on the Application of Computer Vision Based on Deep Learning in Autonomous Driving Technology](https://mlsanity.com/paper/2406.00490/) (2024)
- [Wild2Avatar: Rendering Humans Behind Occlusions](https://mlsanity.com/paper/2401.00431/) (2023)
- [Integrating Medical Imaging and Clinical Reports Using Multimodal Deep Learning for Advanced Disease Analysis](https://mlsanity.com/paper/2405.17459/) (2024)
- [HiPART: Hierarchical Pose AutoRegressive Transformer for Occluded 3D Human Pose Estimation](https://mlsanity.com/paper/2503.23331/) (2025)
- [CHROME: Clothed Human Reconstruction with Occlusion-Resilience and Multiview-Consistency from a Single Image](https://mlsanity.com/paper/2503.15671/) (2025)
- [Investigation of Customized Medical Decision Algorithms Utilizing Graph Neural Networks](https://mlsanity.com/paper/2405.17460/) (2024)
- [Multi-view Pose Fusion for Occlusion-Aware 3D Human Pose Estimation](https://mlsanity.com/paper/2408.15810/) (2024)
- [AJAHR: Amputated Joint Aware 3D Human Mesh Recovery](https://mlsanity.com/paper/2509.19939/) (2025)
- [STRIDE: Single-video based Temporally Continuous Occlusion-Robust 3D Pose Estimation](https://mlsanity.com/paper/2312.16221/) (2023)
- [Di2Pose: Discrete Diffusion Model for Occluded 3D Human Pose Estimation](https://mlsanity.com/paper/2405.17016/) (2024)
- [Application of Vision-Language Model to Pedestrians Behavior and Scene Understanding in Autonomous Driving](https://mlsanity.com/paper/2501.06680/) (2025)
- [Masked Modeling for Human Motion Recovery Under Occlusions](https://mlsanity.com/paper/2601.16079/) (2026)
- [CIGPose: Causal Intervention Graph Neural Network for Whole-Body Pose Estimation](https://mlsanity.com/paper/2603.09418/) (2026)
- [First-place Solution for Streetscape Shop Sign Recognition Competition](https://mlsanity.com/paper/2501.02811/) (2025)
- [DeProPose: Deficiency-Proof 3D Human Pose Estimation via Adaptive Multi-View Fusion](https://mlsanity.com/paper/2502.16419/) (2025)
- [DINeMo: Learning Neural Mesh Models with no 3D Annotations](https://mlsanity.com/paper/2503.20220/) (2025)
- [VOccl3D: A Video Benchmark Dataset for 3D Human Pose and Shape Estimation under real Occlusions](https://mlsanity.com/paper/2508.06757/) (2025)
- [OAHuman: Occlusion-Aware 3D Human Reconstruction from Monocular Images](https://mlsanity.com/paper/2603.14249/) (2026)
- [Discriminative-Generative Synergy for Occlusion Robust 3D Human Mesh Recovery](https://mlsanity.com/paper/2604.21712/) (2026)
- [HumanFlow -- Diffusion-Driven MAV Navigation Among Humans via Tightly-Coupled Motion Tracking, Forecasting, and Control](https://mlsanity.com/paper/2605.25685/) (2026)

## Abstract (arXiv)

Regression-based methods for 3D human pose estimation directly predict the 3D pose parameters from a 2D image using deep networks. While achieving state-of-the-art performance on standard benchmarks, their performance degrades under occlusion. In contrast, optimization-based methods fit a parametric body model to 2D features in an iterative manner. The localized reconstruction loss can potentially make them robust to occlusion, but they suffer from the 2D-3D ambiguity. Motivated by the recent success of generative models in rigid object pose estimation, we propose 3D-aware Neural Body Fitting (3DNBF) - an approximate analysis-by-synthesis approach to 3D human pose estimation with SOTA performance and occlusion robustness. In particular, we propose a generative model of deep features based on a volumetric human representation with Gaussian ellipsoidal kernels emitting 3D pose-dependent feature vectors. The neural features are trained with contrastive learning to become 3D-aware and hence to overcome the 2D-3D ambiguity. Experiments show that 3DNBF outperforms other approaches on both occluded and standard benchmarks. Code is available at https://github.com/edz-o/3DNBF