mlsanityData to 7 Oct 2026

DOVE: Learning Deformable 3D Objects by Watching Videos

arXiv 2107.10844 ↗First posted 22 Jul 2021cs.CVcited by 61

Not on the map. Why?

Abstract (arXiv)

Learning deformable 3D objects from 2D images is often an ill-posed problem. Existing methods rely on explicit supervision to establish multi-view correspondences, such as template shape models and keypoint annotations, which restricts their applicability on objects "in the wild". A more natural way of establishing correspondences is by watching videos of objects moving around. In this paper, we present DOVE, a method that learns textured 3D models of deformable object categories from monocular videos available online, without keypoint, viewpoint or template shape supervision. By resolving symmetry-induced pose ambiguities and leveraging temporal correspondences in videos, the model automatically learns to factor out 3D shape, articulated pose and texture from each individual RGB frame, and is ready for single-image inference at test time. In the experiments, we show that existing methods fail to learn sensible 3D shapes without additional keypoint or template supervision, whereas our method produces temporally consistent 3D models, which can be animated and rendered from arbitrary viewpoints.

Key facts. Cited by 61 papers in the map (data through 7 Oct 2026). Builds on 68 of its 91 references that are in the map. First posted on arXiv 22 Jul 2021.

Builds on

68 of 91 references.

References · 68

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  2. 2017Mask R-CNNcited by 7,259
  3. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  4. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  5. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  6. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
  7. 2018Group Normalizationcited by 1,514
  8. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,459
  9. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,194
  10. 2017End-to-end Recovery of Human Shape and Posecited by 883
  11. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  12. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
38 more
  1. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  2. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 778
  3. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  4. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
  5. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  6. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 649
  7. 2021UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstructioncited by 643
  8. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 587
  9. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  10. 2018AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generationcited by 539
  11. 2020Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Videocited by 470
  12. 2020Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstructioncited by 465
  13. 2019PointRend: Image Segmentation as Renderingcited by 439
  14. 2017Neural 3D Mesh Renderercited by 437
  15. 2019Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoningcited by 420
  16. 2016Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescuecited by 368
  17. 2018Learning Category-Specific Mesh Reconstruction from Image Collectionscited by 317
  18. 2018Learning 3D Human Dynamics from Videocited by 315
  19. 20163D Menagerie: Modeling the 3D shape and pose of animalscited by 310
  20. 2021Neural Articulated Radiance Fieldcited by 196
  21. 2018Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Networkcited by 195
  22. 2019Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wildcited by 180
  23. 2019Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervisioncited by 176
  24. 2016Learning a Predictable and Generative Vector Representation for Objectscited by 173
  25. 2020BlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled Imagescited by 166
  26. 2020Self-supervised Single-view 3D Reconstruction via Semantic Consistencycited by 127
  27. 2019Learning the Depths of Moving People by Watching Frozen Peoplecited by 120
  28. 2021LASR: Learning Articulated Shape Reconstruction from a Monocular Videocited by 115
  29. 2019Exploiting temporal context for 3D human pose estimation in the wildcited by 110
  30. 2019Three-D Safari: Learning to Estimate Zebra Pose, Shape, and Texture from Images "In the Wild"cited by 106
  31. 2019Deep Mesh Reconstruction from Single RGB Images via Topology Modification Networkscited by 106
  32. 2020Image GANs meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Renderingcited by 101
  33. 2019SCOPS: Self-Supervised Co-Part Segmentationcited by 100
  34. 2017Self-supervised Learning of Motion Capturecited by 97
  35. 2019Unsupervised 3D Pose Estimation with Geometric Self-Supervisioncited by 95
  36. 2020Shape and Viewpoint without Keypointscited by 92
  37. 2020Articulation-aware Canonical Surface Mappingcited by 80
  38. 2020Continuous Surface Embeddingscited by 79

50 most cited of 68 All 68 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 10; 2023: 15; 2024: 16; 2025: 8; 2026: 11.

61 papers.

Sort:

By area:

  1. 2021BANMo: Building Animatable 3D Neural Models from Many Casual Videoscited by 195
  2. 2023Consistent4D: Consistent 360deg Dynamic Object Generation from Monocular Videocited by 124
  3. 2023GART: Gaussian Articulated Template Modelscited by 121
  4. 2023Viewset Diffusion: (0-)Image-Conditioned 3D Generative Models from 2D Datacited by 113
  5. 2022MagicPony: Learning Articulated 3D Animals in the Wildcited by 111
  6. 2023HoloDiffusion: Training a 3D Diffusion Model using 2D Imagescited by 106
  7. 2023Visibility Aware Human-Object Interaction Tracking from Single RGB Cameracited by 58
  8. 2024Learning the 3D Fauna of the Webcited by 58
  9. 2022SparsePose: Sparse-View Camera Pose Regression and Refinementcited by 53
  10. 2022BARC: Learning to Regress 3D Dog Shape from Images by Exploiting Breed Informationcited by 51
  11. 2023Reconstructing Animatable Categories from Videoscited by 46
  12. 2024REACTO: Reconstructing Articulated Objects from a Single Videocited by 45
38 more
  1. 2022LaTeRF: Label and Text Driven Object Radiance Fieldscited by 38
  2. 2023Farm3D: Learning Articulated 3D Animals by Distilling 2D Diffusioncited by 37
  3. 2022CASA: Category-agnostic Skeletal Animal Reconstructioncited by 35
  4. 2024DragAPart: Learning a Part-Level Motion Prior for Articulated Objectscited by 32
  5. 2022State of the Art in Dense Monocular Non-Rigid 3D Reconstructioncited by 30
  6. 2022Common Pets in 3D: Dynamic New-View Synthesis of Real-Life Deformable Categoriescited by 29
  7. 2024Recent Trends in 3D Reconstruction of General Non-Rigid Scenescited by 28
  8. 2022Self-Supervised Geometric Correspondence for Category-Level 6D Object Pose Estimation in the Wildcited by 22
  9. 2024Dynamic Gaussians Mesh: Consistent Mesh Reconstruction from Dynamic Scenescited by 22
  10. 2023Ponymation: Learning Articulated 3D Animal Motions from Unlabeled Online Videoscited by 21
  11. 2023Unsupervised Volumetric Animationcited by 18
  12. 2023SAOR: Single-View Articulated Object Reconstructioncited by 15
  13. 2024BAGS: Building Animatable Gaussian Splatting from a Monocular Video with Diffusion Priorscited by 14
  14. 2023MoDA: Modeling Deformable 3D Objects from Casual Videoscited by 13
  15. 2023DreaMo: Articulated 3D Reconstruction From A Single Casual Videocited by 12
  16. 2024DressRecon: Freeform 4D Human Reconstruction from Monocular Videocited by 12
  17. 2023SceNeRFlow: Time-Consistent Reconstruction of General Dynamic Scenescited by 11
  18. 2024DualPM: Dual Posed-Canonical Point Maps for 3D Shape and Pose Reconstructioncited by 10
  19. 2024DreamHOI: Subject-Driven Generation of 3D Human-Object Interactions with Diffusion Priorscited by 9
  20. 2025Common3D: Self-Supervised Learning of 3D Morphable Models for Common Objects in Neural Feature Spacecited by 9
  21. 2022Articulated 3D Human-Object Interactions from RGB Videos: An Empirical Analysis of Approaches and Challengescited by 8
  22. 2023Root Pose Decomposition Towards Generic Non-rigid 3D Reconstruction with Monocular Videoscited by 8
  23. 2025Category-Agnostic Neural Object Riggingcited by 8
  24. 2025Web-Scale Collection of Video Data for 4D Animal Reconstructioncited by 8
  25. 2024Birth and Death of a Rosecited by 7
  26. 20254D-Animal: Freely Reconstructing Animatable 3D Animals from Videoscited by 6
  27. 2024The More You See in 2D, the More You Perceive in 3Dcited by 5
  28. 2025LIM: Large Interpolator Model for Dynamic Reconstructioncited by 5
  29. 2025Model-based Metric 3D Shape and Motion Reconstruction of Wild Bottlenose Dolphins in Drone-Shot Videoscited by 4
  30. 2025DogMo: A Large-Scale Multi-View RGB-D Dataset for 4D Canine Motion Recoverycited by 4
  31. 2023Virtual Pets: Animatable Animal Generation in 3D Scenescited by 3
  32. 2024Agent-to-Sim: Learning Interactive Behavior Models from Casual Longitudinal Videoscited by 3
  33. 2024Towards Multi-Modal Animal Pose Estimation: A Survey and In-Depth Analysiscited by 2
  34. 2025Advances and Trends in the 3D Reconstruction of the Shape and Motion of Animalscited by 2
  35. 20264DEquine: Disentangling Motion and Appearance for 4D Equine Reconstruction from Monocular Videocited by 2
  36. 2026SAM 3D Animal: Promptable Animal 3D Reconstruction from Images in the Wildcited by 2
  37. 2022Learning-based Monocular 3D Reconstruction of Birds: A Contemporary Surveycited by 1
  38. 2024Dense Matchers for Dense Trackingcited by 1

50 most cited of 61 All 61 (JSON)