mlsanityData to 7 Oct 2026

HuMMan: Multi-Modal 4D Human Dataset for Versatile Sensing and Modeling

arXiv 2204.13686 ↗First posted 28 Apr 2022cs.CVcited by 115

Not on the map. Why?

Abstract (arXiv)

4D human sensing and modeling are fundamental tasks in vision and graphics with numerous applications. With the advances of new sensors and algorithms, there is an increasing demand for more versatile datasets. In this work, we contribute HuMMan, a large-scale multi-modal 4D human dataset with 1000 human subjects, 400k sequences and 60M frames. HuMMan has several appealing properties: 1) multi-modal data and annotations including color images, point clouds, keypoints, SMPL parameters, and textured meshes; 2) popular mobile device is included in the sensor suite; 3) a set of 500 actions, designed to cover fundamental movements; 4) multiple tasks such as action recognition, pose estimation, parametric human recovery, and textured mesh reconstruction are supported and evaluated. Extensive experiments on HuMMan voice the need for further study on challenges such as fine-grained action recognition, dynamic human mesh reconstruction, point cloud-based parametric human recovery, and cross-device domain gaps.

Key facts. Cited by 115 papers in the map (data through 7 Oct 2026). Builds on 80 of its 120 references that are in the map. First posted on arXiv 28 Apr 2022.

Builds on

80 of 120 references.

References · 80

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2018SlowFast Networks for Video Recognitioncited by 1,862
  3. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
  4. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  5. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,208
  6. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,199
  7. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,139
  8. 2016Stacked Hourglass Networks for Human Pose Estimationcited by 913
  9. 2017End-to-end Recovery of Human Shape and Posecited by 883
  10. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  11. 2020Convolutional Occupancy Networkscited by 760
  12. 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 692
38 more
  1. 2016NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysiscited by 683
  2. 2020X3D: Expanding Architectures for Efficient Video Recognitioncited by 635
  3. 2020Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humanscited by 634
  4. 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 625
  5. 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 615
  6. 2019NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understandingcited by 530
  7. 2016Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervisioncited by 493
  8. 2021AI Choreographer: Music Conditioned 3D Dance Generation with AIST++cited by 479
  9. 2017AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actionscited by 449
  10. 2017A simple yet effective baseline for 3d human pose estimationcited by 401
  11. 20183D human pose estimation in video with temporal convolutions and semi-supervised trainingcited by 401
  12. 2017Cascaded Pyramid Network for Multi-Person Pose Estimationcited by 341
  13. 2021PARE: Part Attention Regressor for 3D Human Body Estimationcited by 334
  14. 2018Learning 3D Human Dynamics from Videocited by 315
  15. 2019A Short Note on the Kinetics-700 Human Action Datasetcited by 308
  16. 2020HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimationcited by 304
  17. 2019Video Classification with Channel-Separated Convolutional Networkscited by 290
  18. 2020I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Imagecited by 274
  19. 2019Deferred Neural Rendering: Image Synthesis using Neural Texturescited by 253
  20. 2021Function4D: Real-time Human Volumetric Capture from Very Sparse Consumer RGBD Sensorscited by 246
  21. 2018Video Based Reconstruction of 3D People Modelscited by 237
  22. 2020FineGym: A Hierarchical Video Dataset for Fine-grained Action Understandingcited by 232
  23. 2018Learning to Estimate 3D Human Pose and Shape from a Single Color Imagecited by 226
  24. 2018Moments in Time Dataset: one million videos for event understandingcited by 215
  25. 2019Semantic Graph Convolutional Networks for 3D Human Pose Regressioncited by 204
  26. 2021AGORA: Avatars in Geography Optimized for Regression Analysiscited by 203
  27. 2020STAR: Sparse Trained Articulated Human Body Regressorcited by 194
  28. 2020Long-term Human Motion Prediction with Scene Contextcited by 191
  29. 2018Neural Body Fitting: Unifying Deep Learning and Model-Based Human Pose and Shape Estimationcited by 185
  30. 2019DeepHuman: 3D Human Reconstruction from a Single Imagecited by 182
  31. 2020Whole-Body Human Pose Estimation in the Wildcited by 178
  32. 2017PoseTrack: A Benchmark for Human Pose Estimation and Trackingcited by 173
  33. 2018Deep Appearance Models for Face Renderingcited by 164
  34. 2020DeepCap: Monocular Human Performance Capture Using Weak Supervisioncited by 162
  35. 2020Beyond Static Features for Temporally Consistent 3D Human Pose and Shape from a Videocited by 162
  36. 2019Learnable Triangulation of Human Posecited by 160
  37. 2014Cross-view Action Modeling, Learning and Recognitioncited by 134
  38. 2020Combining Implicit Function Learning and Parametric Models for 3D Human Reconstructioncited by 133

50 most cited of 80 All 80 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 10; 2023: 32; 2024: 31; 2025: 27; 2026: 15.

115 papers.

Sort:

By area:

  1. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Modelcited by 768
  2. 2023BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motioncited by 229
  3. 2023Motion-X: A Large-scale 3D Expressive Whole-body Human Motion Datasetcited by 220
  4. 2023PF-LRM: Pose-Free Large Reconstruction Model for Joint Pose and Shape Predictioncited by 165
  5. 2023SMPLer-X: Scaling Up Expressive Human Pose and Shape Estimationcited by 154
  6. 2023One-Stage 3D Whole-Body Mesh Recovery with Component Aware Transformercited by 128
  7. 2022EVA3D: Compositional 3D Human Generation from 2D Image Collectionscited by 111
  8. 2024Nymeria: A Massive Collection of Multimodal Egocentric Daily Motion in the Wildcited by 109
  9. 2023FineMoGen: Fine-Grained Spatio-Temporal Motion Generation and Editingcited by 105
  10. 2023HumanSD: A Native Skeleton-Guided Diffusion Model for Human Image Generationcited by 104
  11. 2023TeCH: Text-guided Reconstruction of Lifelike Clothed Humanscited by 104
  12. 2023DNA-Rendering: A Diverse Neural Actor Repository for High-Fidelity Human-centric Renderingcited by 103
38 more
  1. 2023Human Motion Generation: A Surveycited by 95
  2. 2023SHERF: Generalizable Human NeRF from a Single Imagecited by 88
  3. 2023HumanMAC: Masked Motion Completion for Human Motion Predictioncited by 81
  4. 2023MM-Fi: Multi-Modal Non-Intrusive 4D Human Dataset for Versatile Wireless Sensingcited by 77
  5. 2023High-fidelity 3D Human Digitization from Single 2K Resolution Imagescited by 70
  6. 2024Neural Localizer Fields for Continuous 3D Human Pose and Shape Estimationcited by 57
  7. 2022AvatarCLIP: Zero-Shot Text-Driven Generation and Animation of 3D Avatarscited by 55
  8. 2024IDOL: Instant Photorealistic 3D Human Creation from a Single Imagecited by 55
  9. 2023SynBody: Synthetic Dataset with Layered Human Models for 3D Human Perception and Modelingcited by 54
  10. 2023MVHumanNet: A Large-scale Dataset of Multi-view Daily Dressing Human Capturescited by 54
  11. 2023C·ASE: Learning Conditional Adversarial Skill Embeddings for Physics-based Characterscited by 53
  12. 2023OMG: Towards Open-vocabulary Motion Generation via Mixture of Controllerscited by 47
  13. 2022Being Comes from Not-being: Open-vocabulary Text-to-Motion Generation with Wordless Trainingcited by 45
  14. 2022TORE: Token Reduction for Efficient Human Mesh Recovery with Transformercited by 41
  15. 2023Zolly: Zoom Focal Length Correctly for Perspective-Distorted Human Mesh Reconstructioncited by 41
  16. 2024AiOS: All-in-One-Stage Expressive Human Pose and Shape Estimationcited by 41
  17. 2024EgoExo-Fitness: Towards Egocentric and Exocentric Full-Body Action Understandingcited by 35
  18. 2022Text2Human: Text-Driven Controllable Human Image Generationcited by 31
  19. 2024Reconstructing People, Places, and Camerascited by 31
  20. 2022State of the Art in Dense Monocular Non-Rigid 3D Reconstructioncited by 30
  21. 2024WHAC: World-grounded Humans and Camerascited by 29
  22. 2024Generalizable Human Gaussians from Single-View Imagecited by 29
  23. 2024Unimotion: Unifying 3D Human Motion Synthesis and Understandingcited by 29
  24. 2022Benchmarking and Analyzing 3D Human Pose and Shape Estimation Beyond Algorithmscited by 28
  25. 2024Learning from Massive Human Videos for Universal Humanoid Pose Controlcited by 27
  26. 2023Revisit Human-Scene Interaction via Space Occupancycited by 24
  27. 2022FLAG3D: A 3D Fitness Activity Dataset with Language Instructioncited by 23
  28. 2024Template-Free Single-View 3D Human Digitalization with Diffusion-Guided LRMcited by 23
  29. 2023NICP: Neural ICP for 3D Human Registration at Scalecited by 21
  30. 2024CloSe: A 3D Clothing Segmentation Dataset and Modelcited by 21
  31. 2024LiveHPS: LiDAR-based Scene-level Human Pose and Shape Estimation in Free Environmentcited by 20
  32. 2024PKU-DyMVHumans: A Multi-View Video Benchmark for High-Fidelity Dynamic Human Modelingcited by 17
  33. 2023Towards Multi-Layered 3D Garments Animationcited by 16
  34. 2024RELI11D: A Comprehensive Multimodal Human Motion Dataset and Methodcited by 16
  35. 2024Text-controlled Motion Mamba: Text-Instructed Temporal Grounding of Human Motioncited by 16
  36. 2025Compositional Generative Model of Unbounded 4D Citiescited by 16
  37. 2025EgoSim: An Egocentric Multi-view Simulator and Real Dataset for Body-worn Cameras during Motion and Activitycited by 15
  38. 2025HAMSt3R: Human-Aware Multi-view Stereo 3D Reconstructioncited by 15

50 most cited of 115 All 115 (JSON)