mlsanityData to 14 Oct 2026

To The Point: Correspondence-driven monocular 3D category reconstruction

arXiv 2106.05662 ↗First posted 10 Jun 2021cs.CVcited by 21

Not on the map. Why?

Abstract (arXiv)

We present To The Point (TTP), a method for reconstructing 3D objects from a single image using 2D to 3D correspondences learned from weak supervision. We recover a 3D shape from a 2D image by first regressing the 2D positions corresponding to the 3D template vertices and then jointly estimating a rigid camera transform and non-rigid template deformation that optimally explain the 2D positions through the 3D shape projection. By relying on 3D-2D correspondences we use a simple per-sample optimization problem to replace CNN-based regression of camera pose and non-rigid deformation and thereby obtain substantially more accurate 3D reconstructions. We treat this optimization as a differentiable layer and train the whole system in an end-to-end manner. We report systematic quantitative improvements on multiple categories and provide qualitative results comprising diverse shape, pose and texture prediction examples. Project website: https://fkokkinos.github.io/to_the_point/.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Builds on 33 of its 68 references that are in the map. First posted on arXiv 10 Jun 2021.

Builds on

33 of 68 references.

References · 33

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  2. 2017Mask R-CNNcited by 7,277
  3. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 784
  4. 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 695
  5. 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 627
  6. 2016Learning to learn by gradient descent by gradient descentcited by 620
  7. 2019Deep Equilibrium Modelscited by 615
  8. 2018DensePose: Dense Human Pose Estimation In The Wildcited by 575
  9. 2019Differentiable Convex Optimization Layerscited by 465
  10. 2018Learning Category-Specific Mesh Reconstruction from Image Collectionscited by 317
  11. 2016DSAC - Differentiable RANSAC for Camera Localizationcited by 266
  12. 2019Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wildcited by 180
21 more
  1. 2017Multi-view Supervision for Single-view Reconstruction via Differentiable Ray Consistencycited by 177
  2. 2020Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wildcited by 146
  3. 2020Self-supervised Single-view 3D Reconstruction via Semantic Consistencycited by 127
  4. 2020Exemplar Fine-Tuning for 3D Human Model Fitting Towards In-the-Wild 3D Human Pose Estimationcited by 113
  5. 2017Self-supervised Learning of Motion Capturecited by 97
  6. 2020Articulation-aware Canonical Surface Mappingcited by 81
  7. 2019Deep Declarative Networks: A New Hopecited by 78
  8. 2019End-to-End Learnable Geometric Vision by Backpropagating PnP Optimizationcited by 65
  9. 2019Canonical Surface Mapping via Geometric Cycle Consistencycited by 63
  10. 2019C3DPO: Canonical 3D Pose Networks for Non-Rigid Structure From Motioncited by 61
  11. 2014Category-Specific Object Reconstruction from a Single Imagecited by 56
  12. 2015Sparseness Meets Deepness: 3D Human Pose Estimation from Monocular Videocited by 54
  13. 2020Online Adaptation for Consistent Mesh Reconstruction in the Wildcited by 51
  14. 2019Deep Non-Rigid Structure from Motioncited by 38
  15. 2020Solving the Blind Perspective-n-Point Problem End-To-End With Robust Differentiable Geometric Optimizationcited by 38
  16. 2018Iterative Joint Image Demosaicking and Denoising using a Residual Denoising Networkcited by 26
  17. 2019Lifting AutoEncoders: Unsupervised Learning of a Fully-Disentangled 3D Morphable Model using Deep Non-Rigid Structure from Motioncited by 23
  18. 2021Learning monocular 3D reconstruction of articulated categories from motioncited by 18
  19. 2015Sparse Representation for 3D Shape Estimation: A Convex Relaxation Approachcited by 16
  20. 2020Self-Supervised Multi-View Synchronization Learning for 3D Pose Estimationcited by 9
  21. 2020Canonical 3D Deformer Maps: Unifying parametric and non-parametric methods for dense weakly-supervised category reconstructioncited by 6

Built on by

Cited by year

Citing papers per year: 2021: 4; 2022: 6; 2023: 5; 2024: 3; 2025: 3.

21 papers.

Sort:

By area:

  1. 2021BANMo: Building Animatable 3D Neural Models from Many Casual Videoscited by 197
  2. 2022MagicPony: Learning Articulated 3D Animals in the Wildcited by 112
  3. 2021DOVE: Learning Deformable 3D Objects by Watching Videoscited by 61
  4. 2022Neural Surface Reconstruction of Dynamic Scenes with Monocular RGB-D Cameracited by 57
  5. 2022SparsePose: Sparse-View Camera Pose Regression and Refinementcited by 53
  6. 2023Reconstructing Animatable Categories from Videoscited by 47
  7. 2023Farm3D: Learning Articulated 3D Animals by Distilling 2D Diffusioncited by 38
  8. 2022State of the Art in Dense Monocular Non-Rigid 3D Reconstructioncited by 30
  9. 2023Ponymation: Learning Articulated 3D Animal Motions from Unlabeled Online Videoscited by 21
  10. 2024Distillation of Diffusion Features for Semantic Correspondencecited by 17
  11. 2021Learning to Fit Morphable Modelscited by 16
  12. 2023SAOR: Single-View Articulated Object Reconstructioncited by 16
9 more
  1. 2024LEIA: Latent View-invariant Embeddings for Implicit 3D Articulationcited by 16
  2. 2024BAGS: Building Animatable Gaussian Splatting from a Monocular Video with Diffusion Priorscited by 14
  3. 2025Do It Yourself: Learning Semantic Correspondence from Pseudo-Labelscited by 13
  4. 2023DreaMo: Articulated 3D Reconstruction From A Single Casual Videocited by 12
  5. 2025Common3D: Self-Supervised Learning of 3D Morphable Models for Common Objects in Neural Feature Spacecited by 9
  6. 2025LIM: Large Interpolator Model for Dynamic Reconstructioncited by 5
  7. 2021End-to-End Learning of Multi-category 3D Pose and Shape Estimationcited by 1
  8. 2022Learning-based Monocular 3D Reconstruction of Birds: A Contemporary Surveycited by 1
  9. 2022Piecewise Planar Hulls for Semi-Supervised Learning of 3D Shape and Pose from 2D Imagescited by 0