mlsanityData to 14 Oct 2026

EPOCH: Jointly Estimating the 3D Pose of Cameras and Humans

arXiv 2406.19726 ↗First posted 28 Jun 2024cs.CV cs.GR cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Monocular Human Pose Estimation (HPE) aims at determining the 3D positions of human joints from a single 2D image captured by a camera. However, a single 2D point in the image may correspond to multiple points in 3D space. Typically, the uniqueness of the 2D-3D relationship is approximated using an orthographic or weak-perspective camera model. In this study, instead of relying on approximations, we advocate for utilizing the full perspective camera model. This involves estimating camera parameters and establishing a precise, unambiguous 2D-3D relationship. To do so, we introduce the EPOCH framework, comprising two main components: the pose lifter network (LiftNet) and the pose regressor network (RegNet). LiftNet utilizes the full perspective camera model to precisely estimate the 3D pose in an unsupervised manner. It takes a 2D pose and camera parameters as inputs and produces the corresponding 3D pose estimation. These inputs are obtained from RegNet, which starts from a single image and provides estimates for the 2D pose and camera parameters. RegNet utilizes only 2D pose data as weak supervision. Internally, RegNet predicts a 3D pose, which is then projected to 2D using the estimated camera parameters. This process enables RegNet to establish the unambiguous 2D-3D relationship. Our experiments show that modeling the lifting as an unsupervised task with a camera in-the-loop results in better generalization to unseen data. We obtain state-of-the-art results for the 3D HPE on the Human3.6M and MPI-INF-3DHP datasets. Our code is available at: [Github link upon acceptance, see supplementary materials].

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 44 of its 59 references that are in the map. First posted on arXiv 28 Jun 2024.

Builds on

44 of 59 references.

References · 44

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  2. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  3. 2016Density estimation using Real NVPcited by 2,042
  4. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,545
  5. 2016Stacked Hourglass Networks for Human Pose Estimationcited by 915
  6. 2016Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervisioncited by 494
  7. 2022ViTPose: Simple Vision Transformer Baselines for Human Pose Estimationcited by 456
  8. 2017A simple yet effective baseline for 3d human pose estimationcited by 401
  9. 2017Cascaded Pyramid Network for Multi-Person Pose Estimationcited by 341
  10. 2022CLIFF: Carrying Location Information in Full Frames into Human Pose and Shape Estimationcited by 238
  11. 2016Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Posecited by 221
  12. 2020Deep Learning-Based Human Pose Estimation: A Surveycited by 207
32 more
  1. 2022MotionBERT: A Unified Perspective on Learning Human Motion Representationscited by 193
  2. 2022MixSTE: Seq2seq Mixed Spatio-Temporal Encoder for 3D Human Pose Estimation in Videocited by 159
  3. 2021MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimationcited by 151
  4. 2021SPEC: Seeing People in the Wild with an Estimated Cameracited by 135
  5. 2021Human Pose Regression with Residual Log-likelihood Estimationcited by 131
  6. 2020TransPose: Keypoint Localization via Transformercited by 126
  7. 2017Compositional Human Pose Regressioncited by 124
  8. 2018Ordinal Depth Supervision for 3D Human Pose Estimationcited by 107
  9. 2019Self-Supervised Learning of 3D Human Pose using Multi-view Geometrycited by 98
  10. 2019Unsupervised 3D Pose Estimation with Geometric Self-Supervisioncited by 95
  11. 2021PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimationcited by 89
  12. 2019RepNet: Weakly Supervised Training of an Adversarial Reprojection Network for 3D Human Pose Estimationcited by 85
  13. 2019Stacked Capsule Autoencoderscited by 82
  14. 2023Diffusion-Based 3D Human Pose Estimation with Multi-Hypothesis Aggregationcited by 82
  15. 2018Unsupervised Geometry-Aware Representation for 3D Human Pose Estimationcited by 73
  16. 2018Learning Monocular 3D Human Pose Estimation from Multi-view Imagescited by 70
  17. 2020CanonPose: Self-Supervised Monocular 3D Human Pose Estimation in the Wildcited by 65
  18. 2021Recent Advances in Monocular 2D and 3D Human Pose Estimation: A Deep Learning Perspectivecited by 50
  19. 2019In the Wild Human Pose Estimation Using Explicit 2D Features and Intermediate 3D Representationscited by 48
  20. 2018Can 3D Pose be Learned from 2D Projections Alone?cited by 44
  21. 2020Self-Supervised 3D Human Pose Estimation via Part Guided Novel Image Synthesiscited by 44
  22. 2021ElePose: Unsupervised 3D Human Pose Estimation by Predicting Camera Elevation and Learning Normalizing Flows on 2D Posescited by 35
  23. 2021AdaptPose: Cross-Dataset Adaptation for 3D Human Pose Estimation by Learnable Motion Generationcited by 32
  24. 2020Beyond Weak Perspective for Monocular 3D Human Pose Estimationcited by 29
  25. 2017Adversarial Inverse Graphics Networks: Learning 2D-to-3D Lifting and Image-to-Image Translation from Unpaired Supervisioncited by 27
  26. 2020Unsupervised part representation by Flow Capsulescited by 25
  27. 2022Uncertainty-Aware Adaptation for Self-Supervised 3D Human Pose Estimationcited by 25
  28. 2022Learnable human mesh triangulation for 3D human pose and shape estimationcited by 16
  29. 2022Occluded Human Body Capture with Self-Supervised Spatial-Temporal Motion Priorcited by 15
  30. 2021MetaPose: Fast 3D Pose from Multiple Views without 3D Supervisioncited by 12
  31. 2021Unsupervised 3D Pose Estimation for Hierarchical Dance Video Recognitioncited by 9
  32. 2022Non-Local Latent Relation Distillation for Self-Adaptive 3D Human Pose Estimationcited by 6

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025PersPose: 3D Human Pose Estimation with Perspective Encoding and Perspective Rotationcited by 3