Articulation-aware Canonical Surface Mapping
Not on the map. Why?
Abstract (arXiv)
We tackle the tasks of: 1) predicting a Canonical Surface Mapping (CSM) that indicates the mapping from 2D pixels to corresponding points on a canonical template shape, and 2) inferring the articulation and pose of the template corresponding to the input image. While previous approaches rely on keypoint supervision for learning, we present an approach that can learn without such annotations. Our key insight is that these tasks are geometrically related, and we can obtain supervisory signal via enforcing consistency among the predictions. We present results across a diverse set of animal object categories, showing that our method can learn articulation and CSM prediction from image collections using only foreground mask labels for training. We empirically show that allowing articulation helps learn more accurate CSM prediction, and that enforcing the consistency with predicted CSM is similarly critical for learning meaningful articulation.
Key facts. Cited by 80 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Apr 2020.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 25; 2022: 14; 2023: 18; 2024: 12; 2025: 6; 2026: 5.
80 papers.
Sort:
By area:
- 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
- 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 646
- 2023Diffusion Hyperfeatures: Searching Through Time and Space for Semantic Correspondencecited by 211
- 2022Monocular Dynamic View Synthesis: A Reality Checkcited by 199
- 2021BANMo: Building Animatable 3D Neural Models from Many Casual Videoscited by 195
- 2021LASR: Learning Articulated Shape Reconstruction from a Monocular Videocited by 115
- 2021A-SDF: Learning Disentangled Signed Distance Functions for Articulated Shape Representationcited by 113
- 2022MagicPony: Learning Articulated 3D Animals in the Wildcited by 111
- 2021PixelSynth: Generating a 3D-Consistent Experience from a Single Imagecited by 82
- 2023Multiview Compressive Coding for 3D Reconstructioncited by 71
- 2021Unsupervised Learning of 3D Object Categories from Videos in the Wildcited by 67
- 2021Shelf-Supervised Mesh Prediction in the Wildcited by 65
38 more
- 2022LASSIE: Learning Articulated Shapes from Sparse Image Ensemble via 3D Part Discoverycited by 65
- 2021DOVE: Learning Deformable 3D Objects by Watching Videoscited by 61
- 2021GAN-Supervised Dense Visual Alignmentcited by 61
- 2024Learning the 3D Fauna of the Webcited by 58
- 2021UnsupervisedR&R: Unsupervised Point Cloud Registration via Differentiable Renderingcited by 44
- 2021Learning Generative Models of Textured 3D Meshes from Real-World Imagescited by 44
- 2023Animal3D: A Comprehensive Dataset of 3D Animal Pose and Shapecited by 43
- 2023Farm3D: Learning Articulated 3D Animals by Distilling 2D Diffusioncited by 37
- 2022Understanding 3D Object Articulation in Internet Videoscited by 36
- 2022CASA: Category-agnostic Skeletal Animal Reconstructioncited by 35
- 2022State of the Art in Dense Monocular Non-Rigid 3D Reconstructioncited by 30
- 2022Hi-LASSIE: High-Fidelity Articulated Shape and Skeleton Discovery from Sparse Image Ensemblecited by 30
- 2021Differentiable Stereopsis: Meshes from multiple views using differentiable renderingcited by 26
- 2022Pre-train, Self-train, Distill: A simple recipe for Supersizing 3D Reconstructioncited by 26
- 2023ASIC: Aligning Sparse in-the-wild Image Collectionscited by 26
- 2024Real3D: Scaling Up Large Reconstruction Models with Real-World Imagescited by 24
- 2022Self-Supervised Geometric Correspondence for Category-Level 6D Object Pose Estimation in the Wildcited by 22
- 2023ARTIC3D: Learning Robust Articulated 3D Shapes from Noisy Web Image Collectionscited by 22
- 2021To The Point: Correspondence-driven monocular 3D category reconstructioncited by 21
- 2022Neural Correspondence Field for Object Pose Estimationcited by 21
- 2023Ponymation: Learning Articulated 3D Animal Motions from Unlabeled Online Videoscited by 21
- 2022Learning 3D Object Shape and Layout without 3D Supervisioncited by 19
- 2021Learning monocular 3D reconstruction of articulated categories from motioncited by 18
- 2023OmniMotionGPT: Animal Motion Generation with Limited Datacited by 18
- 2021Fully Understanding Generic Objects: Modeling, Segmentation, and Reconstructioncited by 16
- 2024LEIA: Latent View-invariant Embeddings for Implicit 3D Articulationcited by 16
- 2023SAOR: Single-View Articulated Object Reconstructioncited by 15
- 2024Survey on Modeling of Human-made Articulated Objectscited by 14
- 2021Discovering Relationships between Object Categories via Universal Canonical Mapscited by 13
- 2023MoDA: Modeling Deformable 3D Objects from Casual Videoscited by 13
- 2025Feature4X: Bridging Any Monocular Video to 4D Agentic AI with Versatile Gaussian Feature Fieldscited by 13
- 2025Detection Based Part-level Articulated Object Reconstruction from Single RGBD Imagecited by 13
- 2021Unsupervised Pose-Aware Part Decomposition for 3D Articulated Objectscited by 12
- 2023DreaMo: Articulated 3D Reconstruction From A Single Casual Videocited by 12
- 20222D GANs Meet Unsupervised Single-view 3D Reconstructioncited by 11
- 2024S3O: A Dual-Phase Approach for Reconstructing Dynamic Shape and Skeleton of Articulated Objects from Single Monocular Videocited by 11
- 2024SHIC: Shape-Image Correspondences with no Keypoint Supervisioncited by 11
- 2023CAMM: Building Category-Agnostic and Animatable 3D Models from Monocular Videoscited by 10
50 most cited of 80 All 80 (JSON)
- 2026Kirin: Animal Motion Generation from In-the-Wild Video1 Sep 2026
- 2026BAT3R: Bootstrapping Articulated 3D Reconstruction from 2D Image Collections4 Jul 2026
- 2026Progressive Pose-Guided 4D Animal Reconstruction from Monocular Video30 Jun 2026
- 2026PRIMA: Boosting Animal Mesh Recovery with Biological Priors and Test-Time Adaptation1 Jun 2026
- 2026Best Segmentation Buddies for Image-Shape Correspondence18 May 2026
- 2025Web-Scale Collection of Video Data for 4D Animal Reconstruction3 Nov 2025
- 2025Advances and Trends in the 3D Reconstruction of the Shape and Motion of Animals22 Aug 2025
- 2025Common3D: Self-Supervised Learning of 3D Morphable Models for Common Objects in Neural Feature Space30 Apr 2025
- 2025Predict-Optimize-Distill: A Self-Improving Cycle for 4D Object Understanding24 Apr 2025
- 2025Detection Based Part-level Articulated Object Reconstruction from Single RGBD Image4 Apr 2025
- 2025Feature4X: Bridging Any Monocular Video to 4D Agentic AI with Versatile Gaussian Feature Fields26 Mar 2025
- 2024DualPM: Dual Posed-Canonical Point Maps for 3D Shape and Pose Reconstruction5 Dec 2024
38 more
- 2024Reconstructing Animals and the Wild27 Nov 2024
- 20244DPV: 4D Pet from Videos by Coarse-to-Fine Non-Rigid Radiance Fields15 Nov 2024
- 2024Dessie: Disentanglement for Articulated 3D Horse Shape and Pose Estimation from Images4 Oct 2024
- 2024LEIA: Latent View-invariant Embeddings for Implicit 3D Articulation10 Sep 2024
- 2024SHIC: Shape-Image Correspondences with no Keypoint Supervision26 Jul 2024
- 2024Real3D: Scaling Up Large Reconstruction Models with Real-World Images12 Jun 2024
- 2024S3O: A Dual-Phase Approach for Reconstructing Dynamic Shape and Skeleton of Articulated Objects from Single Monocular Video21 May 2024
- 2024WALT3D: Generating Realistic Training Data from Time-Lapse Imagery for Reconstructing Dynamic Objects under Occlusion27 Mar 2024
- 2024Survey on Modeling of Human-made Articulated Objects22 Mar 2024
- 2024Learning Implicit Representation for Reconstructing Articulated Objects16 Jan 2024
- 2024Learning the 3D Fauna of the Web4 Jan 2024
- 2023Ponymation: Learning Articulated 3D Animal Motions from Unlabeled Online Videos21 Dec 2023
- 2023DreaMo: Articulated 3D Reconstruction From A Single Casual Video5 Dec 2023
- 2023OmniMotionGPT: Animal Motion Generation with Limited Data30 Nov 2023
- 2023LRM: Large Reconstruction Model for Single Image to 3D8 Nov 2023
- 2023Animal3D: A Comprehensive Dataset of 3D Animal Pose and Shape22 Aug 2023
- 2023ARTIC3D: Learning Robust Articulated 3D Shapes from Noisy Web Image Collections7 Jun 2023
- 2023Diffusion Hyperfeatures: Searching Through Time and Space for Semantic Correspondence23 May 2023
- 2023Learning Articulated Shape with Keypoint Pseudo-labels from Web Images27 Apr 2023
- 2023Farm3D: Learning Articulated 3D Animals by Distilling 2D Diffusion20 Apr 2023
- 2023MoDA: Modeling Deformable 3D Objects from Casual Videos17 Apr 2023
- 2023CAMM: Building Category-Agnostic and Animatable 3D Models from Monocular Videos14 Apr 2023
- 2023ASIC: Aligning Sparse in-the-wild Image Collections28 Mar 2023
- 2023SAOR: Single-View Articulated Object Reconstruction23 Mar 2023
- 2023Normal-guided Garment UV Prediction for Human Re-texturing11 Mar 2023
- 2023Canonical mapping as a general-purpose object descriptor for robotic manipulation2 Mar 2023
- 2023Self-supervised learning of object pose estimation using keypoint prediction14 Feb 2023
- 2023Multiview Compressive Coding for 3D Reconstruction19 Jan 2023
- 2023CA2T-Net: Category-Agnostic 3D Articulation Transfer from Single Image5 Jan 2023
- 2022Hi-LASSIE: High-Fidelity Articulated Shape and Skeleton Discovery from Sparse Image Ensemble21 Dec 2022
- 2022MagicPony: Learning Articulated 3D Animals in the Wild22 Nov 2022
- 2022CASA: Category-agnostic Skeletal Animal Reconstruction4 Nov 2022
- 2022State of the Art in Dense Monocular Non-Rigid 3D Reconstruction27 Oct 2022
- 2022Monocular Dynamic View Synthesis: A Reality Check24 Oct 2022
- 2022Self-Supervised Geometric Correspondence for Category-Level 6D Object Pose Estimation in the Wild13 Oct 2022
- 2022Neural Correspondence Field for Object Pose Estimation30 Jul 2022
- 20222D GANs Meet Unsupervised Single-view 3D Reconstruction20 Jul 2022
- 2022Learning-based Monocular 3D Reconstruction of Birds: A Contemporary Survey10 Jul 2022
50 newest of 80 All 80 (JSON)
- 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
- 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 646
- 2022Monocular Dynamic View Synthesis: A Reality Checkcited by 199
- 2021BANMo: Building Animatable 3D Neural Models from Many Casual Videoscited by 195
- 2021LASR: Learning Articulated Shape Reconstruction from a Monocular Videocited by 115
- 2021A-SDF: Learning Disentangled Signed Distance Functions for Articulated Shape Representationcited by 113
- 2022MagicPony: Learning Articulated 3D Animals in the Wildcited by 111
- 2021PixelSynth: Generating a 3D-Consistent Experience from a Single Imagecited by 82
- 2023Multiview Compressive Coding for 3D Reconstructioncited by 71
- 2021Unsupervised Learning of 3D Object Categories from Videos in the Wildcited by 67
- 2021Shelf-Supervised Mesh Prediction in the Wildcited by 65
- 2022LASSIE: Learning Articulated Shapes from Sparse Image Ensemble via 3D Part Discoverycited by 65
12 most cited of 77 All 77 (JSON)
- 2026Kirin: Animal Motion Generation from In-the-Wild Video1 Sep 2026
- 2026BAT3R: Bootstrapping Articulated 3D Reconstruction from 2D Image Collections4 Jul 2026
- 2026Progressive Pose-Guided 4D Animal Reconstruction from Monocular Video30 Jun 2026
- 2026PRIMA: Boosting Animal Mesh Recovery with Biological Priors and Test-Time Adaptation1 Jun 2026
- 2026Best Segmentation Buddies for Image-Shape Correspondence18 May 2026
- 2025Web-Scale Collection of Video Data for 4D Animal Reconstruction3 Nov 2025
- 2025Advances and Trends in the 3D Reconstruction of the Shape and Motion of Animals22 Aug 2025
- 2025Common3D: Self-Supervised Learning of 3D Morphable Models for Common Objects in Neural Feature Space30 Apr 2025
- 2025Predict-Optimize-Distill: A Self-Improving Cycle for 4D Object Understanding24 Apr 2025
- 2025Detection Based Part-level Articulated Object Reconstruction from Single RGBD Image4 Apr 2025
- 2025Feature4X: Bridging Any Monocular Video to 4D Agentic AI with Versatile Gaussian Feature Fields26 Mar 2025
- 2024DualPM: Dual Posed-Canonical Point Maps for 3D Shape and Pose Reconstruction5 Dec 2024
12 newest of 77 All 77 (JSON)