mlsanityData to 7 Oct 2026

Effective Whole-body Pose Estimation with Two-stages Distillation

arXiv 2307.15880 ↗First posted 29 Jul 2023cs.CVcited by 307

Not on the map. Why?

Cited by 86 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Whole-body pose estimation localizes the human body, hand, face, and foot keypoints in an image. This task is challenging due to multi-scale body parts, fine-grained localization for low-resolution regions, and data scarcity. Meanwhile, applying a highly efficient and accurate pose estimator to widely human-centric understanding and generation tasks is urgent. In this work, we present a two-stage pose Distillation for Whole-body Pose estimators, named DWPose, to improve their effectiveness and efficiency. The first-stage distillation designs a weight-decay strategy while utilizing a teacher's intermediate feature and final logits with both visible and invisible keypoints to supervise the student from scratch. The second stage distills the student model itself to further improve performance. Different from the previous self-knowledge distillation, this stage finetunes the student's head with only 20% training time as a plug-and-play training strategy. For data limitations, we explore the UBody dataset that contains diverse facial expressions and hand gestures for real-life applications. Comprehensive experiments show the superiority of our proposed simple yet effective methods. We achieve new state-of-the-art performance on COCO-WholeBody, significantly boosting the whole-body AP of RTMPose-l from 64.8% to 66.5%, even surpassing RTMPose-x teacher with 65.3% AP. We release a series of models with different sizes, from tiny to large, for satisfying various downstream tasks. Our codes and models are available at https://github.com/IDEA-Research/DWPose.

Key facts. Cited by 307 papers in the map (data through 7 Oct 2026). Builds on 64 of its 71 references that are in the map. First posted on arXiv 29 Jul 2023.

Builds on

64 of 71 references.

References · 64

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  7. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  8. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  9. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,339
  10. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,161
  11. 2021YOLOX: Exceeding YOLO Series in 2021cited by 1,064
  12. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,061
38 more
  1. 2022On Distillation of Guided Diffusion Modelscited by 721
  2. 2019MediaPipe: A Framework for Building Perception Pipelinescited by 572
  3. 2018Simple Baselines for Human Pose Estimation and Trackingcited by 502
  4. 2020GRAB: A Dataset of Whole-Body Human Grasping of Objectscited by 397
  5. 2022Decoupled Knowledge Distillationcited by 380
  6. 2022ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulationcited by 306
  7. 2013DeepPose: Human Pose Estimation via Deep Neural Networkscited by 305
  8. 2019FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Imagescited by 271
  9. 2020InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Imagecited by 244
  10. 2023BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motioncited by 229
  11. 2023Motion-X: A Large-scale 3D Expressive Whole-body Human Motion Datasetcited by 220
  12. 2016Associative Embedding: End-to-End Learning for Joint Detection and Groupingcited by 211
  13. 2022Knowledge Distillation from A Stronger Teachercited by 204
  14. 2023DepGraph: Towards Any Structural Pruningcited by 202
  15. 2022AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Timecited by 199
  16. 2022Generating Holistic 3D Human Motion from Speechcited by 180
  17. 2020Whole-Body Human Pose Estimation in the Wildcited by 178
  18. 2022BEAT: A Large-Scale Semantic and Emotional Multi-Modal Dataset for Conversational Gestures Synthesiscited by 177
  19. 2023RTMPose: Real-Time Multi-Person Pose Estimation based on MMPosecited by 170
  20. 2020Channel-wise Knowledge Distillation for Dense Predictioncited by 161
  21. 2020MediaPipe Hands: On-device Real-time Hand Trackingcited by 157
  22. 2021Focal and Global Knowledge Distillation for Detectorscited by 138
  23. 2023One-Stage 3D Whole-Body Mesh Recovery with Component Aware Transformercited by 128
  24. 2021Rethinking Soft Labels for Knowledge Distillation: A Bias-Variance Tradeoff Perspectivecited by 116
  25. 2022Masked Generative Distillationcited by 107
  26. 2023HumanSD: A Native Skeleton-Guided Diffusion Model for Human Image Generationcited by 104
  27. 2022Not All Tokens Are Equal: Human-centric Visual Analysis via Token Clustering Transformercited by 99
  28. 2023AvatarReX: Real-time Expressive Full-body Avatarscited by 83
  29. 2020Accurate 3D Hand Pose Estimation for Whole-Body 3D Human Mesh Estimationcited by 81
  30. 2023TaleCrafter: Interactive Story Visualization with Multiple Characterscited by 71
  31. 2023From Knowledge Distillation to Self-Knowledge Distillation: A Unified Approach with Normalized Loss and Customized Soft Labelscited by 69
  32. 2021SimCC: a Simple Coordinate Classification Perspective for Human Pose Estimationcited by 68
  33. 2022BEVDistill: Cross-Modal BEV Distillation for Multi-View 3D Object Detectioncited by 66
  34. 2018Facial Landmark Detection: a Literature Surveycited by 63
  35. 2021Online Knowledge Distillation for Efficient Pose Estimationcited by 55
  36. 2023Explicit Box Detection Unifies End-to-End Multi-Person Pose Estimationcited by 48
  37. 2022Better Teacher Better Student: Dynamic Prior Knowledge for Knowledge Distillationcited by 41
  38. 2022DAMO-YOLO : A Report on Real-Time Object Detection Designcited by 38

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 9; 2024: 80; 2025: 132; 2026: 86.

307 papers.

Sort:

By area:

  1. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
  2. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
  3. 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animationcited by 683
  4. 2025VACE: All-in-One Video Creation and Editingcited by 467
  5. 2023MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Modelcited by 330
  6. 2024Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidancecited by 271
  7. 2024Sapiens: Foundation for Human Vision Modelscited by 229
  8. 2024MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidancecited by 196
  9. 2024Follow-Your-Emoji: Fine-Controllable and Expressive Freestyle Portrait Animationcited by 172
  10. 2024Loopy: Taming Audio-Driven Portrait Avatar with Long-Term Motion Dependencycited by 102
  11. 2024UniAnimate: Taming Unified Video Diffusion Models for Consistent Human Image Animationcited by 101
  12. 2024MOFA-Video: Controllable Image Animation via Generative Motion Field Adaptions in Frozen Image-to-Video Diffusion Modelcited by 87
38 more
  1. 2024Animate-X: Universal Character Image Animation with Enhanced Motion Representationcited by 86
  2. 2024StableAnimator: High-Quality Identity-Preserving Human Image Animationcited by 85
  3. 2025Autoregressive Adversarial Post-Training for Real-Time Interactive Video Generationcited by 84
  4. 2024OpenHumanVid: A Large-Scale High-Quality Dataset for Enhancing Human-Centric Video Generationcited by 77
  5. 2025FantasyTalking: Realistic Talking Portrait Generation via Coherent Motion Synthesiscited by 77
  6. 2024GEM: A Generalizable Ego-Vision Multimodal World Model for Fine-Grained Ego-Motion, Object Dynamics, and Scene Composition Controlcited by 76
  7. 2024EchoMimicV2: Towards Striking, Simplified, and Semi-Body Human Animationcited by 69
  8. 2024HumanVid: Demystifying Training Data for Camera-controllable Human Image Animationcited by 63
  9. 2025Wan-S2V: Audio-Driven Cinematic Video Generationcited by 60
  10. 2025HunyuanPortrait: Implicit Condition Control for Enhanced Portrait Animationcited by 55
  11. 2024PhysAvatar: Learning the Physics of Dressed 3D Avatars from Visual Observationscited by 51
  12. 2023LEMON: Learning 3D Human-Object Interaction Relation from 2D Imagescited by 49
  13. 2024AnimateDiff-Lightning: Cross-Model Diffusion Distillationcited by 47
  14. 2024Wear-Any-Way: Manipulable Virtual Try-on via Sparse Correspondence Alignmentcited by 42
  15. 2025Follow-Your-Emoji-Faster: Towards Efficient, Fine-Controllable, and Expressive Freestyle Portrait Animationcited by 41
  16. 2024FitDiT: Advancing the Authentic Garment Details for High-fidelity Virtual Try-oncited by 39
  17. 2024CyberHost: Taming Audio-driven Avatar Diffusion Model with Region Codebook Attentioncited by 38
  18. 2025HumanDiT: Pose-Guided Diffusion Transformer for Long-form Human Motion Video Generationcited by 38
  19. 2024RealisDance: Equip controllable character animation with realistic handscited by 37
  20. 2024DisPose: Disentangling Pose Guidance for Controllable Human Image Animationcited by 35
  21. 2024Co-Speech Gesture Video Generation via Motion-Decoupled Diffusion Modelcited by 34
  22. 2025InstructX: Towards Unified Visual Editing with MLLM Guidancecited by 31
  23. 2026FireRed-Image-Edit-1.0 Technical Reportcited by 31
  24. 2024MotionFollower: Editing Video Motion via Lightweight Score-Guided Diffusioncited by 29
  25. 2025RealisDance-DiT: Simple yet Strong Baseline towards Controllable Character Animation in the Wildcited by 28
  26. 2024VividPose: Advancing Stable Video Diffusion for Realistic Human Image Animationcited by 27
  27. 2024MegActor: Harness the Power of Raw Video for Vivid Portrait Animationcited by 27
  28. 2025SpeakerVid-5M: A Large-Scale High-Quality Dataset for Audio-Visual Dyadic Interactive Human Generationcited by 27
  29. 2025Human Motion Video Generation: A Surveycited by 25
  30. 2024A Comprehensive Survey on Human Video Generation: Challenges, Methods, and Insightscited by 24
  31. 2025RynnVLA-001: Using Human Demonstrations to Improve Robot Manipulationcited by 24
  32. 2023DreaMoving: A Human Video Generation Framework based on Diffusion Modelscited by 23
  33. 2024Generative Inbetweening through Frame-wise Conditions-Driven Video Generationcited by 23
  34. 2023Hulk: A Universal Knowledge Translator for Human-Centric Taskscited by 22
  35. 2024AnyFit: Controllable Virtual Try-on for Any Combination of Attire Across Any Scenariocited by 22
  36. 2025GUAVA: Generalizable Upper Body 3D Gaussian Avatarcited by 22
  37. 2024Expressive Gaussian Human Avatars from Monocular RGB Videocited by 21
  38. 2025InterActHuman: Multi-Concept Human Animation with Layout-Aligned Audio Conditionscited by 20

50 most cited of 307 All 307 (JSON)