mlsanityData to 14 Oct 2026

Mirror-Aware Neural Humans

arXiv 2309.04750 ↗First posted 9 Sep 2023cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Human motion capture either requires multi-camera systems or is unreliable when using single-view input due to depth ambiguities. Meanwhile, mirrors are readily available in urban environments and form an affordable alternative by recording two views with only a single camera. However, the mirror setting poses the additional challenge of handling occlusions of real and mirror image. Going beyond existing mirror approaches for 3D human pose estimation, we utilize mirrors for learning a complete body model, including shape and dense appearance. Our main contributions are extending articulated neural radiance fields to include a notion of a mirror, making it sample-efficient over potential occlusion regions. Together, our contributions realize a consumer-level 3D motion capture system that starts from off-the-shelf 2D poses by automatically calibrating the camera, estimating mirror orientation, and subsequently lifting 2D keypoint detections to 3D skeleton pose that is used to condition the mirror-aware NeRF. We empirically demonstrate the benefit of learning a body model and accounting for occlusion in challenging mirror scenes.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 37 of its 57 references that are in the map. First posted on arXiv 9 Sep 2023.

Builds on

37 of 57 references.

References · 37

  1. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
  2. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,214
  3. 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 627
  4. 2016Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervisioncited by 494
  5. 2022HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Videocited by 483
  6. 2022ViTPose: Simple Vision Transformer Baselines for Human Pose Estimationcited by 456
  7. 20183D human pose estimation in video with temporal convolutions and semi-supervised trainingcited by 403
  8. 2016RMPE: Regional Multi-person Pose Estimationcited by 340
  9. 2021A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Posecited by 233
  10. 2022AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Timecited by 200
  11. 2021BANMo: Building Animatable 3D Neural Models from Many Casual Videoscited by 197
  12. 2019Learning to Reconstruct People in Clothing from a Single RGB Cameracited by 168
25 more
  1. 2021Robust High-Resolution Video Matting with Temporal Guidancecited by 137
  2. 2017VNect: Real-time 3D Human Pose Estimation with a Single RGB Cameracited by 129
  3. 2019Self-Supervised Learning of 3D Human Pose using Multi-view Geometrycited by 98
  4. 2020Weakly Supervised 3D Human Pose and Shape Reconstruction with Normalizing Flowscited by 94
  5. 2018LiveCap: Real-time Human Performance Capture from Monocular Videocited by 93
  6. 2021PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimationcited by 89
  7. 2019RepNet: Weakly Supervised Training of an Adversarial Reprojection Network for 3D Human Pose Estimationcited by 85
  8. 2018Unsupervised Geometry-Aware Representation for 3D Human Pose Estimationcited by 73
  9. 2018Learning Monocular 3D Human Pose Estimation from Multi-view Imagescited by 70
  10. 2020Weakly-Supervised 3D Human Pose Learning via Multi-view Images in the Wildcited by 66
  11. 2020CanonPose: Self-Supervised Monocular 3D Human Pose Estimation in the Wildcited by 65
  12. 2019In the Wild Human Pose Estimation Using Explicit 2D Features and Intermediate 3D Representationscited by 48
  13. 2019View-Invariant Probabilistic Embedding for Human Posecited by 47
  14. 2021Reconstructing 3D Human Pose by Watching Humans in the Mirrorcited by 47
  15. 2020Self-Supervised 3D Human Pose Estimation via Part Guided Novel Image Synthesiscited by 44
  16. 2021Bilevel Online Adaptation for Out-of-Domain Human Mesh Reconstructioncited by 39
  17. 2022DANBO: Disentangled Articulated Neural Body Representations via Graph Neural Networkscited by 38
  18. 2021ElePose: Unsupervised 3D Human Pose Estimation by Predicting Camera Elevation and Learning Normalizing Flows on 2D Posescited by 35
  19. 2020Unsupervised 3D Human Pose Representation with Viewpoint and Pose Disentanglementcited by 32
  20. 2021AdaptPose: Cross-Dataset Adaptation for 3D Human Pose Estimation by Learnable Motion Generationcited by 32
  21. 2019Multiview-Consistent Semi-Supervised Learning for 3D Human Pose Estimationcited by 31
  22. 2019Neural Scene Decomposition for Multi-Person Motion Capturecited by 21
  23. 2019Distill Knowledge from NRSfM for Weakly Supervised 3D Pose Learningcited by 13
  24. 2019Weakly-Supervised 3D Pose Estimation from a Single Image using Multi-View Consistencycited by 8
  25. 2021Single View Physical Distance Estimation using Human Posecited by 5

Built on by

Not cited yet.