mlsanityData to 14 Oct 2026

Learning the Depths of Moving People by Watching Frozen People

arXiv 1904.11111 ↗First posted 25 Apr 2019cs.CVcited by 120

Not on the map. Why?

Abstract (arXiv)

We present a method for predicting dense depth in scenarios where both a monocular camera and people in the scene are freely moving. Existing methods for recovering depth for dynamic, non-rigid objects from monocular video impose strong assumptions on the objects' motion and may only recover sparse depth. In this paper, we take a data-driven approach and learn human depth priors from a new source of data: thousands of Internet videos of people imitating mannequins, i.e., freezing in diverse, natural poses, while a hand-held camera tours the scene. Because people are stationary, training data can be generated using multi-view stereo reconstruction. At inference time, our method uses motion parallax cues from the static areas of the scenes to guide the depth prediction. We demonstrate our method on real-world sequences of complex human actions captured by a moving hand-held camera, show improvement over state-of-the-art monocular depth prediction methods, and show various 3D effects produced using our predicted depth.

Key facts. Cited by 120 papers in the map (data through 14 Oct 2026). First posted on arXiv 25 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 30; 2022: 35; 2023: 28; 2024: 13; 2025: 8; 2026: 6.

120 papers.

Sort:

By area:

  1. 2021Vision Transformers for Dense Predictioncited by 1,803
  2. 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 460
  3. 2023TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinementcited by 287
  4. 2023PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Trackingcited by 276
  5. 2022TAP-Vid: A Benchmark for Tracking Any Point in a Videocited by 263
  6. 2024Shape of Motion: 4D Reconstruction from a Single Videocited by 250
  7. 2024MegaSaM: Accurate, Fast, and Robust Structure and Motion from Casual Dynamic Videoscited by 228
  8. 2022DynIBaR: Neural Dynamic Image-Based Renderingcited by 224
  9. 2021AGORA: Avatars in Geography Optimized for Regression Analysiscited by 205
  10. 2023Robust Dynamic Radiance Fieldscited by 194
  11. 2021NerfingMVS: Guided Optimization of Neural Radiance Fields for Indoor Multi-view Stereocited by 192
  12. 2021The Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depthcited by 190
38 more
  1. 2022Fourier PlenOctrees for Dynamic Radiance Field Rendering in Real-timecited by 165
  2. 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Modelscited by 161
  3. 2022NeRF: Neural Radiance Field in 3D Vision: A Comprehensive Review (Updated Post-Gaussian Splatting)cited by 158
  4. 2021Probabilistic Modeling for Human Mesh Recoverycited by 153
  5. 2021Editable Free-viewpoint Video Using a Layered Neural Representationcited by 148
  6. 2023DynamicStereo: Consistent Dynamic Depth from Stereo Videoscited by 148
  7. 2021SPEC: Seeing People in the Wild with an Estimated Cameracited by 135
  8. 2024Flash3D: Feed-Forward Generalisable 3D Scene Reconstruction from a Single Imagecited by 116
  9. 2022Recovering 3D Human Mesh from Monocular Images: A Surveycited by 103
  10. 2023Generative Image Dynamicscited by 94
  11. 2024Stereo4D: Learning How Things Move in 3D from Internet Stereo Videoscited by 89
  12. 2023Neural Residual Radiance Fields for Streamably Free-Viewpoint Videoscited by 77
  13. 2022A Survey on Long-Tailed Visual Recognitioncited by 75
  14. 2021Unsupervised Scale-consistent Depth Learning from Videocited by 69
  15. 2021DOVE: Learning Deformable 3D Objects by Watching Videoscited by 61
  16. 2021360MonoDepth: High-Resolution 360deg Monocular Depth Estimationcited by 60
  17. 2021Consistent Depth of Moving Objects in Videocited by 56
  18. 2021High-Resolution Optical Flow from 1D Attention and Correlationcited by 54
  19. 2022SC-DepthV3: Robust Self-supervised Monocular Depth Estimation for Dynamic Scenescited by 52
  20. 2021SLIDE: Single Image 3D Photography with Soft Layering and Depth-aware Inpaintingcited by 48
  21. 2022Disentangling Object Motion and Occlusion for Unsupervised Multi-frame Monocular Depthcited by 48
  22. 2022Towards Accurate Reconstruction of 3D Scene Shape from A Single Monocular Imagecited by 42
  23. 2022Open-World Instance Segmentation: Exploiting Pseudo Ground Truth From Learned Pairwise Affinitycited by 41
  24. 2024Score-Guided Diffusion for 3D Human Recoverycited by 39
  25. 2021MonoIndoor: Towards Good Practice of Self-Supervised Monocular Depth Estimation for Indoor Environmentscited by 38
  26. 2023Uncertainty-aware State Space Transformer for Egocentric 3D Hand Trajectory Forecastingcited by 34
  27. 2023PACE: Human and Camera Motion Estimation from in-the-wild Videoscited by 34
  28. 2021Instance-wise Occlusion and Depth Orders in Natural Scenescited by 31
  29. 2022Virtual Correspondence: Humans as a Cue for Extreme-View Geometrycited by 25
  30. 2021Beyond Image to Depth: Improving Depth Prediction using Echoescited by 24
  31. 2022BodySLAM: Joint Camera Localisation, Mapping, and Human Motion Trackingcited by 24
  32. 2023VideoRF: Rendering Dynamic Radiance Fields as 2D Feature Video Streamscited by 22
  33. 2023Ponymation: Learning Articulated 3D Animal Motions from Unlabeled Online Videoscited by 21
  34. 2021Moving in a 360 World: Synthesizing Panoramic Parallaxes from a Single Panoramacited by 19
  35. 2023Scene-Aware 3D Multi-Human Motion Capture from a Single Cameracited by 19
  36. 2023MACARONS: Mapping And Coverage Anticipation with RGB Online Self-Supervisioncited by 19
  37. 2022SelfD: Self-Learning Large-Scale Driving Policies From the Webcited by 18
  38. 2022CSFlow: Learning Optical Flow via Cross Strip Correlation for Autonomous Drivingcited by 16

50 most cited of 120 All 120 (JSON)