# Self-Supervised Human Depth Estimation from Monocular Videos

arXiv: https://arxiv.org/abs/2005.03358 · HTML: https://mlsanity.com/paper/2005.03358/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 May 2020.

## Built on by (most cited first, 17 of 17)

- [High-fidelity 3D Human Digitization from Single 2K Resolution Images](https://mlsanity.com/paper/2303.15108/) (2023)
- [Registering Explicit to Implicit: Towards High-Fidelity Garment mesh Reconstruction from Single Images](https://mlsanity.com/paper/2203.15007/) (2022)
- [AvatarBooth: High-Quality and Customizable 3D Human Avatar Generation](https://mlsanity.com/paper/2306.09864/) (2023)
- [Detailed Avatar Recovery from Single Image](https://mlsanity.com/paper/2108.02931/) (2021)
- [GarVerseLOD: High-Fidelity 3D Garment Reconstruction from a Single In-the-Wild Image using a Dataset with Levels of Details](https://mlsanity.com/paper/2411.03047/) (2024)
- [Self-supervised 3D Representation Learning of Dressed Humans from Social Media Videos](https://mlsanity.com/paper/2103.03319/) (2021)
- [SHARP: Shape-Aware Reconstruction of People in Loose Clothing](https://mlsanity.com/paper/2205.11948/) (2022)
- [Self-Supervised Tracking via Target-Aware Data Synthesis](https://mlsanity.com/paper/2106.10900/) (2021)
- [Fusing the Old with the New: Learning Relative Camera Pose with Geometry-Guided Uncertainty](https://mlsanity.com/paper/2104.08278/) (2021)
- [DH-AUG: DH Forward Kinematics Model Driven Augmentation for 3D Human Pose Estimation](https://mlsanity.com/paper/2207.09303/) (2022)
- [Human as Points: Explicit Point-based 3D Human Reconstruction from Single-view RGB Images](https://mlsanity.com/paper/2311.02892/) (2023)
- [RAFaRe: Learning Robust and Accurate Non-parametric 3D Face Reconstruction from Pseudo 2D&3D Pairs](https://mlsanity.com/paper/2302.05486/) (2023)
- [Spatial-Temporal Graph Mamba for Music-Guided Dance Video Synthesis](https://mlsanity.com/paper/2507.06689/) (2025)
- [PIFu for the Real World: A Self-supervised Framework to Reconstruct Dressed Human from Single-view Images](https://mlsanity.com/paper/2208.10769/) (2022)
- [Sketch2PoseNet: Efficient and Generalized Sketch to 3D Human Pose Prediction](https://mlsanity.com/paper/2510.26196/) (2025)
- [Reconstruction of People in Loose Clothing](https://mlsanity.com/paper/2106.04778/) (2021)
- [Monocular Human Digitization via Implicit Re-projection Networks](https://mlsanity.com/paper/2205.06468/) (2022)

## Abstract (arXiv)

Previous methods on estimating detailed human depth often require supervised training with `ground truth' depth data. This paper presents a self-supervised method that can be trained on YouTube videos without known depth, which makes training data collection simple and improves the generalization of the learned network. The self-supervised learning is achieved by minimizing a photo-consistency loss, which is evaluated between a video frame and its neighboring frames warped according to the estimated depth and the 3D non-rigid motion of the human body. To solve this non-rigid motion, we first estimate a rough SMPL model at each video frame and compute the non-rigid body motion accordingly, which enables self-supervised learning on estimating the shape details. Experiments demonstrate that our method enjoys better generalization and performs much better on data in the wild.