mlsanityData to 7 Oct 2026

H3WB: Human3.6M 3D WholeBody Dataset and Benchmark

arXiv 2211.15692 ↗First posted 28 Nov 2022cs.CVcited by 23

Not on the map. Why?

Abstract (arXiv)

We present a benchmark for 3D human whole-body pose estimation, which involves identifying accurate 3D keypoints on the entire human body, including face, hands, body, and feet. Currently, the lack of a fully annotated and accurate 3D whole-body dataset results in deep networks being trained separately on specific body parts, which are combined during inference. Or they rely on pseudo-groundtruth provided by parametric body models which are not as accurate as detection based methods. To overcome these issues, we introduce the Human3.6M 3D WholeBody (H3WB) dataset, which provides whole-body annotations for the Human3.6M dataset using the COCO Wholebody layout. H3WB comprises 133 whole-body keypoint annotations on 100K images, made possible by our new multi-view pipeline. We also propose three tasks: i) 3D whole-body pose lifting from 2D complete whole-body pose, ii) 3D whole-body pose lifting from 2D incomplete whole-body pose, and iii) 3D whole-body pose estimation from a single RGB image. Additionally, we report several baselines from popular methods for these tasks. Furthermore, we also provide automated 3D whole-body annotations of TotalCapture and experimentally show that when used with H3WB it helps to improve the performance. Code and dataset is available at https://github.com/wholebody3d/wholebody3d

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 65 of its 88 references that are in the map. First posted on arXiv 28 Nov 2022.

Builds on

65 of 88 references.

References · 65

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  5. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
  6. 2016Stacked Hourglass Networks for Human Pose Estimationcited by 913
  7. 2017End-to-end Recovery of Human Shape and Posecited by 883
  8. 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 692
  9. 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 625
  10. 2016Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervisioncited by 493
  11. 2022TEMOS: Generating diverse human motions from textual descriptionscited by 486
  12. 2019Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Setcited by 481
38 more
  1. 2017A simple yet effective baseline for 3d human pose estimationcited by 401
  2. 2017Cascaded Pyramid Network for Multi-Person Pose Estimationcited by 341
  3. 2019FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Imagescited by 271
  4. 2019Convolutional Mesh Regression for Single-Image Human Shape Reconstructioncited by 270
  5. 2020InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Imagecited by 244
  6. 2018Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodiescited by 239
  7. 2017Learning to Estimate 3D Hand Pose from Single RGB Imagescited by 234
  8. 2016Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Posecited by 220
  9. 2017Unite the People: Closing the Loop Between 3D and 2D Human Representationscited by 205
  10. 2017Single-Shot Multi-Person 3D Pose Estimation From Monocular RGBcited by 195
  11. 2018Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Networkcited by 195
  12. 20193D Hand Shape and Pose from Images in the Wildcited by 190
  13. 2021SCANimate: Weakly Supervised Learning of Skinned Clothed Avatar Networkscited by 181
  14. 2020Whole-Body Human Pose Estimation in the Wildcited by 178
  15. 2019Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervisioncited by 176
  16. 20193D Hand Shape and Pose Estimation from a Single RGB Imagecited by 174
  17. 2021FrankMocap: A Monocular 3D Whole-Body Pose Estimation System via Regression and Integrationcited by 168
  18. 2018Monocular Total Capture: Posing Face, Body, and Hands in the Wildcited by 162
  19. 2020Beyond Static Features for Temporally Consistent 3D Human Pose and Shape from a Videocited by 162
  20. 2020Monocular Expressive Body Regression through Body-Driven Attentioncited by 155
  21. 2017GANerated Hands for Real-time 3D Hand Tracking from Monocular RGBcited by 147
  22. 2020Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wildcited by 146
  23. 2019Camera Distance-aware Top-down Approach for 3D Multi-person Pose Estimation from a Single RGB Imagecited by 131
  24. 2019End-to-end Hand Mesh Recovery from a Monocular RGB Imagecited by 124
  25. 2015Human Pose Estimation with Iterative Error Feedbackcited by 121
  26. 2020Epipolar Transformerscited by 117
  27. 2019Pushing the Envelope for RGB-based Dense 3D Hand Pose Estimation via Neural Renderingcited by 115
  28. 2018Hand Pose Estimation via Latent 2.5D Heatmap Regressioncited by 111
  29. 2019Exploiting temporal context for 3D human pose estimation in the wildcited by 110
  30. 2016Regressing Robust and Discriminative 3D Morphable Models with a very Deep Neural Networkcited by 109
  31. 2020Weakly Supervised 3D Human Pose and Shape Reconstruction with Normalizing Flowscited by 94
  32. 2018LCR-Net++: Multi-person 2D and 3D Pose Detection in Natural Imagescited by 88
  33. 2020Motion Guided 3D Pose Estimation from Videoscited by 85
  34. 2021Driving-Signal Aware Full-Body Avatarscited by 81
  35. 2021Scene-aware Generative Network for Human Motion Synthesiscited by 70
  36. 2017Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regressioncited by 66
  37. 2020Weakly-Supervised 3D Human Pose Learning via Multi-view Images in the Wildcited by 66
  38. 2020CanonPose: Self-Supervised Monocular 3D Human Pose Estimation in the Wildcited by 65

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 10; 2025: 5; 2026: 7.

23 papers.

Sort:

By area:

  1. 2025HUMOTO: A 4D Dataset of Mocap Human Object Interactionscited by 30
  2. 2024RTMW: Real-Time Multi-Person 2D and 3D Whole-body Pose Estimationcited by 18
  3. 20233D-LFM: Lifting Foundation Modelcited by 8
  4. 2025H2OT: Hierarchical Hourglass Tokenizer for Efficient Video Pose Transformerscited by 5
  5. 2024Motion Capture from Inertial and Vision Sensorscited by 4
  6. 2024VoxelKeypointFusion: Generalizable Multi-View Multi-Person Pose Estimationcited by 3
  7. 2024ShoeModel: Learning to Wear on the User-specified Shoes via Diffusion Modelcited by 2
  8. 2024An Animation-based Augmentation Approach for Action Recognition from Discontinuous Videocited by 2
  9. 2024Quater-GCN: Enhancing 3D Human Pose Estimation with Orientation and Semi-supervised Trainingcited by 2
  10. 2025RapidPoseTriangulation: Multi-view Multi-person Whole-body Human Pose Triangulation in a Millisecondcited by 2
  11. 20243D WholeBody Pose Estimation based on Semantic Graph Attention Network and Distance Informationcited by 1
  12. 2024PAFUSE: Part-based Diffusion for 3D Whole-Body Pose Estimationcited by 1
11 more
  1. 2026SSMNBench: Diagnosing Image-based Cross-View Human-Object Understanding via Single-View Sufficiency and Multi-View Necessitycited by 1
  2. 2024GTAutoAct: An Automatic Datasets Generation Framework Based on Game Engine Redevelopment for Action Recognitioncited by 0
  3. 2024Non-rigid Structure-from-Motion: Temporally-smooth Procrustean Alignment and Spatially-variant Deformation Modelingcited by 0
  4. 20253D Human Pose and Shape Estimation from LiDAR Point Clouds: A Reviewcited by 0
  5. 2025Robotic Classification of Divers' Swimming States using Visual Pose Keypoints as IMUscited by 0
  6. 2026SEAL-pose: Enhancing 3D Human Pose Estimation via a Learned Loss for Structural Consistencycited by 0
  7. 2026A Cookbook of 3D Vision: Data, Learning Paradigms, and Applicationcited by 0
  8. 2026EgoHTR: Egocentric 4D Demonstrations of Human Terrain Traversalcited by 0
  9. 2026Human4K: A Large-Scale 4K Multi-View Mocap Dataset for Whole-Body 3D Human Reconstructioncited by 0
  10. 2026Agent-Centric Animal Pose Forecastingcited by 0
  11. 2026Topology-Unified 2D Pose Estimation across Intact, Residual and Prosthetic Limbscited by 0