mlsanityData to 7 Oct 2026

MPM: A Unified 2D-3D Human Pose Representation via Masked Pose Modeling

arXiv 2306.17201 ↗First posted 29 Jun 2023cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Estimating 3D human poses only from a 2D human pose sequence is thoroughly explored in recent years. Yet, prior to this, no such work has attempted to unify 2D and 3D pose representations in the shared feature space. In this paper, we propose mpm, a unified 2D-3D human pose representation framework via masked pose modeling. We treat 2D and 3D poses as two different modalities like vision and language and build a single-stream transformer-based architecture. We apply two pretext tasks, which are masked 2D pose modeling, and masked 3D pose modeling to pre-train our network and use full-supervision to perform further fine-tuning. A high masking ratio of 71.8~% in total with a spatio-temporal mask sampling strategy leads to better relation modeling both in spatial and temporal domains. mpm~can handle multiple tasks including 3D human pose estimation, 3D pose estimation from occluded 2D pose, and 3D pose completion in a single framework. We conduct extensive experiments and ablation studies on several widely used human pose datasets and achieve state-of-the-art performance on MPI-INF-3DHP.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 46 of its 58 references that are in the map. First posted on arXiv 29 Jun 2023.

Builds on

46 of 58 references.

References · 46

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  8. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  9. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
  10. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,291
  11. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,208
  12. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,199
34 more
  1. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  2. 2016Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervisioncited by 493
  3. 2022Masked Autoencoders As Spatiotemporal Learnerscited by 428
  4. 2017A simple yet effective baseline for 3d human pose estimationcited by 401
  5. 20183D human pose estimation in video with temporal convolutions and semi-supervised trainingcited by 401
  6. 2022Multimodal Learning with Transformers: A Surveycited by 399
  7. 2017Cascaded Pyramid Network for Multi-Person Pose Estimationcited by 341
  8. 2022BEiT v2: Masked Image Modeling with Vector-Quantized Visual Tokenizerscited by 298
  9. 20213D Human Pose Estimation with Spatial and Temporal Transformerscited by 271
  10. 2020ActBERT: Learning Global-Local Video-Text Representationscited by 267
  11. 2020UniVL: A Unified Video and Language Pre-Training Model for Multimodal Understanding and Generationcited by 258
  12. 2022MixSTE: Seq2seq Mixed Spatio-Temporal Encoder for 3D Human Pose Estimation in Videocited by 158
  13. 2021MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimationcited by 151
  14. 20163D Human Pose Estimation = 2D Pose Estimation + Matchingcited by 112
  15. 2020SRNet: Improving Generalization in 3D Human Pose Estimation with a Split-and-Recombine Approachcited by 94
  16. 2021Exploiting Temporal Contexts with Strided Transformer for 3D Human Pose Estimationcited by 94
  17. 2019Generating Multiple Hypotheses for 3D Human Pose Estimation with Mixture Density Networkcited by 92
  18. 2023PoseFormerV2: Exploring Frequency Domain for Efficient and Robust 3D Human Pose Estimationcited by 91
  19. 2021PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimationcited by 88
  20. 2020Motion Guided 3D Pose Estimation from Videoscited by 85
  21. 2020Cascaded deep monocular 3D human pose estimation with evolutionary training datacited by 80
  22. 2022P-STMO: Pre-Trained Spatial Temporal Many-to-One Model for 3D Human Pose Estimationcited by 80
  23. 2022GFPose: Learning 3D Human Pose Prior with Gradient Fieldscited by 57
  24. 2021Conditional Directed Graph Convolution for 3D Human Pose Estimationcited by 52
  25. 2019Symbiotic Graph Neural Networks for 3D Skeleton-based Human Action Recognition and Motion Predictioncited by 50
  26. 2020InterBERT: Vision-and-Language Interaction for Multi-modal Pretrainingcited by 48
  27. 2018Can 3D Pose be Learned from 2D Projections Alone?cited by 44
  28. 2021AdaptPose: Cross-Dataset Adaptation for 3D Human Pose Estimation by Learnable Motion Generationcited by 32
  29. 2019Trajectory Space Factorization for Deep Video-Based 3D Human Pose Estimationcited by 30
  30. 2020Feedback Graph Convolutional Network for Skeleton-based Action Recognitioncited by 22
  31. 2023Global Adaptation meets Local Generalization: Unsupervised Domain Adaptation for 3D Human Pose Estimationcited by 19
  32. 2021Improving Robustness and Accuracy via Relative Information Encoding in 3D Human Pose Estimationcited by 18
  33. 2023DIVOTrack: A Novel Dataset and Baseline Method for Cross-View Multi-Object Tracking in DIVerse Open Scenescited by 17
  34. 2022Weakly Supervised Two-Stage Training Scheme for Deep Video Fight Detection Modelcited by 5

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 1; 2025: 1.

4 papers.

  1. 2024RT-Pose: A 4D Radar Tensor-based 3D Human Pose Estimation and Localization Benchmarkcited by 16
  2. 2023Efficient Domain Adaptation via Generative Prior for 3D Infant Pose Estimationcited by 8
  3. 2023UniHPE: Towards Unified Human Pose Estimation via Contrastive Learningcited by 2
  4. 2025UniHPR: Unified Human Pose Representation via Singular Value Contrastive Learningcited by 1