C3DPO: Canonical 3D Pose Networks for Non-Rigid Structure From Motion
Not on the map. Why?
Abstract (arXiv)
We propose C3DPO, a method for extracting 3D models of deformable objects from 2D keypoint annotations in unconstrained images. We do so by learning a deep network that reconstructs a 3D object from a single view at a time, accounting for partial occlusions, and explicitly factoring the effects of viewpoint changes and object deformations. In order to achieve this factorization, we introduce a novel regularization technique. We first show that the factorization is successful if, and only if, there exists a certain canonicalization function of the reconstructed shapes. Then, we learn the canonicalization function together with the reconstruction one, which constrains the result to be consistent. We demonstrate state-of-the-art reconstruction results for methods that do not use ground-truth 3D supervision for a number of benchmarks, including Up3D and PASCAL3D+. Source code has been made available at https://github.com/facebookresearch/c3dpo_nrsfm.
Key facts. Cited by 60 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Sep 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 18; 2022: 18; 2023: 8; 2024: 4; 2025: 9; 2026: 3.
60 papers.
Sort:
By area:
- 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
- 2023PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Trackingcited by 274
- 2024Shape of Motion: 4D Reconstruction from a Single Videocited by 247
- 2022Particle Video Revisited: Tracking Through Occlusions Using Point Trajectoriescited by 215
- 2021LASR: Learning Articulated Shape Reconstruction from a Monocular Videocited by 115
- 2022RelPose: Predicting Probabilistic Relative Rotation for Single Objects in the Wildcited by 102
- 2025St4RTrack: Simultaneous 4D Reconstruction and Tracking in the Worldcited by 88
- 2023Multiview Compressive Coding for 3D Reconstructioncited by 71
- 2021DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervisioncited by 59
- 2022SparsePose: Sparse-View Camera Pose Regression and Refinementcited by 53
- 2021Watch It Move: Unsupervised Discovery of 3D Joints for Re-Posing of Articulated Objectscited by 50
- 2022Estimating Egocentric 3D Human Pose in the Wild with External Weak Supervisioncited by 33
38 more
- 2022ConDor: Self-Supervised Canonicalization of 3D Pose for Partial Shapescited by 31
- 2021Gravity-Aware Monocular 3D Human-Object Reconstructioncited by 30
- 2022State of the Art in Dense Monocular Non-Rigid 3D Reconstructioncited by 30
- 2021KAMA: 3D Keypoint Aware Body Mesh Articulationcited by 27
- 2022Uncertainty-Aware Adaptation for Self-Supervised 3D Human Pose Estimationcited by 25
- 2022Unbiased 4D: Monocular 4D Reconstruction with a Neural Deformation Modelcited by 25
- 2022φ-SfT: Shape-from-Template with a Physics-Based Deformation Modelcited by 23
- 2021To The Point: Correspondence-driven monocular 3D category reconstructioncited by 21
- 2022Learning 3D Object Shape and Layout without 3D Supervisioncited by 19
- 2022Shape-Pose Disentanglement using SE(3)-equivariant Vector Neuronscited by 16
- 2021StrobeNet: Category-Level Multiview Reconstruction of Articulated Objectscited by 14
- 2024Fast Encoder-Based 3D from Casual Videos via Point Track Processingcited by 14
- 2021Discovering Relationships between Object Categories via Universal Canonical Mapscited by 13
- 2021MoCaNet: Motion Retargeting in-the-wild via Canonicalization Networkscited by 13
- 2021Self-Supervised 3D Human Pose Estimation with Multiple-View Geometrycited by 10
- 2021PAUL: Procrustean Autoencoder for Unsupervised Liftingcited by 9
- 2021PR-RRN: Pairwise-Regularized Residual-Recursive Networks for Non-rigid Structure-from-Motioncited by 9
- 2022Canonical Fields: Self-Supervised Learning of Pose-Canonicalized Neural Fieldscited by 9
- 2022Deep Non-rigid Structure-from-Motion: A Sequence-to-Sequence Translation Perspectivecited by 8
- 20233D-LFM: Lifting Foundation Modelcited by 8
- 2025Dynamic Scene Reconstruction: Recent Advance in Real-time Rendering and Streamingcited by 8
- 2022MHR-Net: Multiple-Hypothesis Reconstruction of Non-Rigid Shapes from 2D Viewscited by 7
- 2023Unsupervised 3D Pose Estimation with Non-Rigid Structure-from-Motion Modelingcited by 7
- 2022Non-Local Latent Relation Distillation for Self-Adaptive 3D Human Pose Estimationcited by 6
- 2021DensePose 3D: Lifting Canonical Surface Maps of Articulated Objects to the Third Dimensioncited by 5
- 2022Unsupervised Learning of 3D Semantic Keypoints with Mutual Reconstructioncited by 5
- 2022Category-Level Global Camera Pose Estimation with Multi-Hypothesis Point Cloud Correspondencescited by 3
- 2023Stable and Consistent Prediction of 3D Characteristic Orientation via Invariant Residual Learningcited by 3
- 2025Ev4DGS: Novel-view Rendering of Non-Rigid Objects from Monocular Event Streamscited by 3
- 2021GTT-Net: Learned Generalized Trajectory Triangulationcited by 2
- 2021High Fidelity 3D Reconstructions with Limited Physical Viewscited by 2
- 20233D Surface Reconstruction in the Wild by Deforming Shape Priors from Synthetic Datacited by 2
- 2026Learning Global Motion with Compact Gaussians for Feed-Forward 4D Reconstructioncited by 2
- 2021Semi-supervised 3D Hand-Object Pose Estimation via Pose Dictionary Learningcited by 1
- 2021Pose Recognition in the Wild: Animal pose estimation using Agglomerative Clustering and Contrastive Learningcited by 1
- 2021End-to-End Learning of Multi-category 3D Pose and Shape Estimationcited by 1
- 2025Non-Rigid Structure-from-Motion via Differential Geometry with Recoverable Conformal Scalecited by 1
- 2025Inferring Compositional 4D Scenes without Ever Seeing Onecited by 1
50 most cited of 60 All 60 (JSON)
- 2026Learning Global Motion with Compact Gaussians for Feed-Forward 4D Reconstruction29 May 2026
- 2026Category-Level 3D Correspondence in Camera Space via Morphable Object Priors27 May 2026
- 2026SSR: A Training-Free Approach for Streaming 3D Reconstruction16 Mar 2026
- 2025Inferring Compositional 4D Scenes without Ever Seeing One4 Dec 2025
- 2025KeyPointDiffuser: Unsupervised 3D Keypoint Learning via Latent Diffusion Models3 Dec 2025
- 2025Gaussians on Fire: High-Frequency Reconstruction of Flames27 Nov 2025
- 2025Ev4DGS: Novel-view Rendering of Non-Rigid Objects from Monocular Event Streams13 Oct 2025
- 2025Non-Rigid Structure-from-Motion via Differential Geometry with Recoverable Conformal Scale2 Oct 2025
- 2025Unsupervised 2D-3D lifting of non-rigid objects using local constraints27 Apr 2025
- 2025St4RTrack: Simultaneous 4D Reconstruction and Tracking in the World17 Apr 2025
- 2025Dynamic Scene Reconstruction: Recent Advance in Real-time Rendering and Streaming11 Mar 2025
- 2025Toward Efficient Generalization in 3D Human Pose Estimation via a Canonical Domain Approach27 Jan 2025
38 more
- 2024Object Agnostic 3D Lifting in Space and Time2 Dec 2024
- 2024Shape of Motion: 4D Reconstruction from a Single Video18 Jul 2024
- 2024Non-rigid Structure-from-Motion: Temporally-smooth Procrustean Alignment and Spatially-variant Deformation Modeling7 May 2024
- 2024Fast Encoder-Based 3D from Casual Videos via Point Track Processing10 Apr 2024
- 20233D-LFM: Lifting Foundation Model19 Dec 2023
- 2023HeadRecon: High-Fidelity 3D Head Reconstruction from Monocular Video14 Dec 2023
- 2023LRM: Large Reconstruction Model for Single Image to 3D8 Nov 2023
- 2023Unsupervised 3D Pose Estimation with Non-Rigid Structure-from-Motion Modeling18 Aug 2023
- 2023PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking27 Jul 2023
- 2023Stable and Consistent Prediction of 3D Characteristic Orientation via Invariant Residual Learning20 Jun 2023
- 20233D Surface Reconstruction in the Wild by Deforming Shape Priors from Synthetic Data24 Feb 2023
- 2023Multiview Compressive Coding for 3D Reconstruction19 Jan 2023
- 2022Canonical Fields: Self-Supervised Learning of Pose-Canonicalized Neural Fields5 Dec 2022
- 2022SparsePose: Sparse-View Camera Pose Regression and Refinement29 Nov 2022
- 2022Piecewise Planar Hulls for Semi-Supervised Learning of 3D Shape and Pose from 2D Images14 Nov 2022
- 2022State of the Art in Dense Monocular Non-Rigid 3D Reconstruction27 Oct 2022
- 2022Category-Level Global Camera Pose Estimation with Multi-Hypothesis Point Cloud Correspondences28 Sep 2022
- 2022RelPose: Predicting Probabilistic Relative Rotation for Single Objects in the Wild11 Aug 2022
- 2022MHR-Net: Multiple-Hypothesis Reconstruction of Non-Rigid Shapes from 2D Views19 Jul 2022
- 2022Unbiased 4D: Monocular 4D Reconstruction with a Neural Deformation Model16 Jun 2022
- 2022Learning 3D Object Shape and Layout without 3D Supervision14 Jun 2022
- 2022Deep Non-rigid Structure-from-Motion: A Sequence-to-Sequence Translation Perspective10 Apr 2022
- 2022Particle Video Revisited: Tracking Through Occlusions Using Point Trajectories8 Apr 2022
- 2022Non-Local Latent Relation Distillation for Self-Adaptive 3D Human Pose Estimation5 Apr 2022
- 2022Shape-Pose Disentanglement using SE(3)-equivariant Vector Neurons3 Apr 2022
- 2022Uncertainty-Aware Adaptation for Self-Supervised 3D Human Pose Estimation29 Mar 2022
- 2022φ-SfT: Shape-from-Template with a Physics-Based Deformation Model22 Mar 2022
- 2022Unsupervised Learning of 3D Semantic Keypoints with Mutual Reconstruction19 Mar 2022
- 2022Estimating Egocentric 3D Human Pose in the Wild with External Weak Supervision20 Jan 2022
- 2022ConDor: Self-Supervised Canonicalization of 3D Pose for Partial Shapes19 Jan 2022
- 2021Watch It Move: Unsupervised Discovery of 3D Joints for Re-Posing of Articulated Objects21 Dec 2021
- 2021End-to-End Learning of Multi-category 3D Pose and Shape Estimation19 Dec 2021
- 2021MoCaNet: Motion Retargeting in-the-wild via Canonicalization Networks19 Dec 2021
- 2021Pose Recognition in the Wild: Animal pose estimation using Agglomerative Clustering and Contrastive Learning16 Nov 2021
- 2021High Fidelity 3D Reconstructions with Limited Physical Views22 Oct 2021
- 2021GTT-Net: Learned Generalized Trajectory Triangulation8 Sep 2021
- 2021DensePose 3D: Lifting Canonical Surface Maps of Articulated Objects to the Third Dimension31 Aug 2021
- 2021Gravity-Aware Monocular 3D Human-Object Reconstruction19 Aug 2021
50 newest of 60 All 60 (JSON)
- 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
- 2023PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Trackingcited by 274
- 2024Shape of Motion: 4D Reconstruction from a Single Videocited by 247
- 2022Particle Video Revisited: Tracking Through Occlusions Using Point Trajectoriescited by 215
- 2021LASR: Learning Articulated Shape Reconstruction from a Monocular Videocited by 115
- 2022RelPose: Predicting Probabilistic Relative Rotation for Single Objects in the Wildcited by 102
- 2025St4RTrack: Simultaneous 4D Reconstruction and Tracking in the Worldcited by 88
- 2023Multiview Compressive Coding for 3D Reconstructioncited by 71
- 2021DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervisioncited by 59
- 2022SparsePose: Sparse-View Camera Pose Regression and Refinementcited by 53
- 2021Watch It Move: Unsupervised Discovery of 3D Joints for Re-Posing of Articulated Objectscited by 50
- 2022Estimating Egocentric 3D Human Pose in the Wild with External Weak Supervisioncited by 33
12 most cited of 59 All 59 (JSON)
- 2026Learning Global Motion with Compact Gaussians for Feed-Forward 4D Reconstruction29 May 2026
- 2026Category-Level 3D Correspondence in Camera Space via Morphable Object Priors27 May 2026
- 2026SSR: A Training-Free Approach for Streaming 3D Reconstruction16 Mar 2026
- 2025Inferring Compositional 4D Scenes without Ever Seeing One4 Dec 2025
- 2025Gaussians on Fire: High-Frequency Reconstruction of Flames27 Nov 2025
- 2025Ev4DGS: Novel-view Rendering of Non-Rigid Objects from Monocular Event Streams13 Oct 2025
- 2025Non-Rigid Structure-from-Motion via Differential Geometry with Recoverable Conformal Scale2 Oct 2025
- 2025Unsupervised 2D-3D lifting of non-rigid objects using local constraints27 Apr 2025
- 2025St4RTrack: Simultaneous 4D Reconstruction and Tracking in the World17 Apr 2025
- 2025Dynamic Scene Reconstruction: Recent Advance in Real-time Rendering and Streaming11 Mar 2025
- 2025Toward Efficient Generalization in 3D Human Pose Estimation via a Canonical Domain Approach27 Jan 2025
- 2024Object Agnostic 3D Lifting in Space and Time2 Dec 2024
12 newest of 59 All 59 (JSON)