# FinePOSE: Fine-Grained Prompt-Driven 3D Human Pose Estimation via Diffusion Models

arXiv: https://arxiv.org/abs/2405.05216 · HTML: https://mlsanity.com/paper/2405.05216/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 52 of its 59 references that are in the map.
- First posted on arXiv 8 May 2024.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 30 of 52)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [Structured Denoising Diffusion Models in Discrete State-Spaces](https://mlsanity.com/paper/2107.03006/) (2021)
- [Conditional Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2203.05557/) (2022)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model](https://mlsanity.com/paper/2208.15001/) (2022)
- [GPT Understands, Too](https://mlsanity.com/paper/2103.10385/) (2021)
- [Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervision](https://mlsanity.com/paper/1611.09813/) (2016)
- [Blended Latent Diffusion](https://mlsanity.com/paper/2206.02779/) (2022)
- [3D human pose estimation in video with temporal convolutions and semi-supervised training](https://mlsanity.com/paper/1811.11742/) (2018)
- [Cascaded Pyramid Network for Multi-Person Pose Estimation](https://mlsanity.com/paper/1711.07319/) (2017)
- [Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.06503/) (2022)
- [Prompt Distribution Learning](https://mlsanity.com/paper/2205.03340/) (2022)
- [MotionBERT: A Unified Perspective on Learning Human Motion Representations](https://mlsanity.com/paper/2210.06551/) (2022)
- [PointCLIP V2: Prompting CLIP and GPT for Powerful 3D Open-world Learning](https://mlsanity.com/paper/2211.11682/) (2022)
- [Video Probabilistic Diffusion Models in Projected Latent Space](https://mlsanity.com/paper/2302.07685/) (2023)
- [Populating 3D Scenes by Learning Human-Scene Interaction](https://mlsanity.com/paper/2012.11581/) (2020)
- [MixSTE: Seq2seq Mixed Spatio-Temporal Encoder for 3D Human Pose Estimation in Video](https://mlsanity.com/paper/2203.00859/) (2022)
- [MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimation](https://mlsanity.com/paper/2111.12707/) (2021)
- [Synthesizing Long-Term 3D Human Motion and Interaction in 3D Scenes](https://mlsanity.com/paper/2012.05522/) (2020)
- [FineDiving: A Fine-grained Dataset for Procedure-aware Action Quality Assessment](https://mlsanity.com/paper/2204.03646/) (2022)
- [DiffPose: Toward More Reliable 3D Pose Estimation](https://mlsanity.com/paper/2211.16940/) (2022)
- [Graph Stacked Hourglass Networks for 3D Human Pose Estimation](https://mlsanity.com/paper/2103.16385/) (2021)

## Built on by (most cited first, 23 of 23)

- [Towards Balanced Multi-Modal Learning in 3D Human Pose Estimation](https://mlsanity.com/paper/2501.05264/) (2025)
- [Single View Garment Reconstruction Using Diffusion Mapping Via Pattern Coordinates](https://mlsanity.com/paper/2504.08353/) (2025)
- [TrackNetV4: Enhancing Fast Sports Object Tracking with Motion Attention Maps](https://mlsanity.com/paper/2409.14543/) (2024)
- [HumanSAM: Classifying Human-centric Forgery Videos in Human Spatial, Appearance, and Motion Anomaly](https://mlsanity.com/paper/2507.19924/) (2025)
- [Benchmarking 3D Human Pose Estimation Models under Occlusions](https://mlsanity.com/paper/2504.10350/) (2025)
- [Human-in-Context: Unified Cross-Domain 3D Human Motion Modeling via In-Context Learning](https://mlsanity.com/paper/2508.10897/) (2025)
- [BLAPose: Enhancing 3D Human Pose Estimation with Bone Length Adjustment](https://mlsanity.com/paper/2410.20731/) (2024)
- [NanoHTNet: Nano Human Topology Network for Efficient 3D Human Pose Estimation](https://mlsanity.com/paper/2501.15763/) (2025)
- [PoseBench3D: A Cross-Dataset Analysis Framework for 3D Human Pose Estimation via Pose Lifting Networks](https://mlsanity.com/paper/2505.10888/) (2025)
- [DreamPose3D: Hallucinative Diffusion with Prompt Learning for 3D Human Pose Estimation](https://mlsanity.com/paper/2511.09502/) (2025)
- [FastDDHPose: Towards Unified, Efficient, and Disentangled 3D Human Pose Estimation](https://mlsanity.com/paper/2512.14162/) (2025)
- [MuPPet: Multi-person 2D-to-3D Pose Lifting](https://mlsanity.com/paper/2604.09715/) (2026)
- [Quality assessment of 3D human animation: Subjective and objective evaluation](https://mlsanity.com/paper/2505.23301/) (2025)
- [StarPose: 3D Human Pose Estimation via Spatial-Temporal Autoregressive Diffusion](https://mlsanity.com/paper/2508.02056/) (2025)
- [Gait Recognition via Collaborating Discriminative and Generative Diffusion Models](https://mlsanity.com/paper/2511.06245/) (2025)
- [Mocap-2-to-3: Multi-view Lifting for Monocular Motion Recovery with 2D Pretraining](https://mlsanity.com/paper/2503.03222/) (2025)
- [HDiffTG: A Lightweight Hybrid Diffusion-Transformer-GCN Architecture for 3D Human Pose Estimation](https://mlsanity.com/paper/2505.04276/) (2025)
- [KASportsFormer: Kinematic Anatomy Enhanced Transformer for 3D Human Pose Estimation on Short Sports Scene Video](https://mlsanity.com/paper/2507.20763/) (2025)
- [HyperDiff: Hypergraph Guided Diffusion Model for 3D Human Pose Estimation](https://mlsanity.com/paper/2508.14431/) (2025)
- [Efficient Diffusion-Based 3D Human Pose Estimation with Hierarchical Temporal Pruning](https://mlsanity.com/paper/2508.21363/) (2025)
- [Spatio-Temporal Garment Reconstruction Using Diffusion Mapping via Pattern Coordinates](https://mlsanity.com/paper/2602.24043/) (2026)
- [HYPERPOSE: Hyperbolic Kinematic Phase-Space Attention for 3D Human Pose Estimation](https://mlsanity.com/paper/2605.10100/) (2026)
- [Quality-Preserving Imperceptible Adversarial Attack on Skeleton-based Human Action Recognition](https://mlsanity.com/paper/2606.13022/) (2026)

## Abstract (arXiv)

The 3D Human Pose Estimation (3D HPE) task uses 2D images or videos to predict human joint coordinates in 3D space. Despite recent advancements in deep learning-based methods, they mostly ignore the capability of coupling accessible texts and naturally feasible knowledge of humans, missing out on valuable implicit supervision to guide the 3D HPE task. Moreover, previous efforts often study this task from the perspective of the whole human body, neglecting fine-grained guidance hidden in different body parts. To this end, we present a new Fine-Grained Prompt-Driven Denoiser based on a diffusion model for 3D HPE, named FinePOSE. It consists of three core blocks enhancing the reverse process of the diffusion model: (1) Fine-grained Part-aware Prompt learning (FPP) block constructs fine-grained part-aware prompts via coupling accessible texts and naturally feasible knowledge of body parts with learnable prompts to model implicit guidance. (2) Fine-grained Prompt-pose Communication (FPC) block establishes fine-grained communications between learned part-aware prompts and poses to improve the denoising quality. (3) Prompt-driven Timestamp Stylization (PTS) block integrates learned prompt embedding and temporal information related to the noise level to enable adaptive adjustment at each denoising step. Extensive experiments on public single-human pose estimation datasets show that FinePOSE outperforms state-of-the-art methods. We further extend FinePOSE to multi-human pose estimation. Achieving 34.3mm average MPJPE on the EgoHumans dataset demonstrates the potential of FinePOSE to deal with complex multi-human scenarios. Code is available at https://github.com/PKU-ICST-MIPL/FinePOSECVPR2024.