mlsanityData to 14 Oct 2026

Rethinking Self-Supervised Visual Representation Learning in Pre-training for 3D Human Pose and Shape Estimation

arXiv 2303.05370 ↗First posted 9 Mar 2023cs.CVcited by 7

Not on the map. Why?

Abstract (arXiv)

Recently, a few self-supervised representation learning (SSL) methods have outperformed the ImageNet classification pre-training for vision tasks such as object detection. However, its effects on 3D human body pose and shape estimation (3DHPSE) are open to question, whose target is fixed to a unique class, the human, and has an inherent task gap with SSL. We empirically study and analyze the effects of SSL and further compare it with other pre-training alternatives for 3DHPSE. The alternatives are 2D annotation-based pre-training and synthetic data pre-training, which share the motivation of SSL that aims to reduce the labeling cost. They have been widely utilized as a source of weak-supervision or fine-tuning, but have not been remarked as a pre-training source. SSL methods underperform the conventional ImageNet classification pre-training on multiple 3DHPSE benchmarks by 7.7% on average. In contrast, despite a much less amount of pre-training data, the 2D annotation-based pre-training improves accuracy on all benchmarks and shows faster convergence during fine-tuning. Our observations challenge the naive application of the current SSL pre-training to 3DHPSE and relight the value of other data types in the pre-training aspect.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 57 of its 67 references that are in the map. First posted on arXiv 9 Mar 2023.

Builds on

57 of 67 references.

References · 57

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  6. 2017Mask R-CNNcited by 7,277
  7. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  8. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  9. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  10. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  11. 2020Supervised Contrastive Learningcited by 2,829
  12. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
38 more
  1. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,162
  2. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
  3. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,380
  4. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  5. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,064
  6. 2017End-to-end Recovery of Human Shape and Posecited by 885
  7. 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 695
  8. 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 627
  9. 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 620
  10. 2018Exploring the Limits of Weakly Supervised Pretrainingcited by 605
  11. 2018DensePose: Dense Human Pose Estimation In The Wildcited by 575
  12. 2018Simple Baselines for Human Pose Estimation and Trackingcited by 502
  13. 2016Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervisioncited by 494
  14. 2020End-to-End Human Pose and Mesh Reconstruction with Transformerscited by 418
  15. 2018Rethinking ImageNet Pre-trainingcited by 377
  16. 2017Learning from Synthetic Humanscited by 359
  17. 2021PARE: Part Attention Regressor for 3D Human Body Estimationcited by 337
  18. 2015Understanding Neural Networks Through Deep Visualizationcited by 334
  19. 2018Learning 3D Human Dynamics from Videocited by 316
  20. 2020I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Imagecited by 274
  21. 2019FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Imagescited by 271
  22. 2019Convolutional Mesh Regression for Single-Image Human Shape Reconstructioncited by 270
  23. 2021PyMAF: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loopcited by 250
  24. 2020Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Posecited by 248
  25. 2021AGORA: Avatars in Geography Optimized for Regression Analysiscited by 205
  26. 2021Self-supervised Pretraining of Visual Features in the Wildcited by 198
  27. 2017Single-Shot Multi-Person 3D Pose Estimation From Monocular RGBcited by 195
  28. 2020Beyond Static Features for Temporally Consistent 3D Human Pose and Shape from a Videocited by 163
  29. 2021Efficient Visual Pretraining with Contrastive Detectioncited by 150
  30. 2020Demystifying Contrastive Self-Supervised Learning: Invariances, Augmentations and Dataset Biasescited by 143
  31. 2021Are Large-scale Datasets Necessary for Self-Supervised Pre-training?cited by 117
  32. 2020Exemplar Fine-Tuning for 3D Human Model Fitting Towards In-the-Wild 3D Human Pose Estimationcited by 113
  33. 2017Extreme clicking for efficient object annotationcited by 81
  34. 2020Accurate 3D Hand Pose Estimation for Whole-Body 3D Human Mesh Estimationcited by 81
  35. 2018Unsupervised Geometry-Aware Representation for 3D Human Pose Estimationcited by 73
  36. 2021Learning to Estimate Robust 3D Human Mesh from In-the-Wild Crowded Scenescited by 65
  37. 2021Playing for 3D Human Recoverycited by 64
  38. 2020NeuralAnnot: Neural Annotator for 3D Human Mesh Training Setscited by 62

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 3; 2024: 2; 2025: 2.

7 papers.

  1. 2024Joint Reconstruction of 3D Human and Object via Contact-Based Refinement Transformercited by 29
  2. 2023Cyclic Test-Time Adaptation on Monocular Video for 3D Human Mesh Reconstructioncited by 18
  3. 2024MEGA: Masked Generative Autoencoder for Human Mesh Recoverycited by 14
  4. 2023Cross-view and Cross-pose Completion for 3D Human Understandingcited by 6
  5. 2023Towards Robust and Expressive Whole-body Human Pose and Shape Estimationcited by 6
  6. 2025DeClotH: Decomposable 3D Cloth and Human Body Reconstruction from a Single Imagecited by 4
  7. 2025Learning Transformation-Isomorphic Latent Space for Accurate Hand Pose Estimationcited by 0