# Discovering Relationships between Object Categories via Universal Canonical Maps

arXiv: https://arxiv.org/abs/2106.09758 · HTML: https://mlsanity.com/paper/2106.09758/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 62 references that are in the map.
- First posted on arXiv 17 Jun 2021.

## Builds on (most cited first, 30 of 32)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [LVIS: A Dataset for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/1908.03195/) (2019)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [End-to-end Recovery of Human Shape and Pose](https://mlsanity.com/paper/1712.06584/) (2017)
- [Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loop](https://mlsanity.com/paper/1909.12828/) (2019)
- [DensePose: Dense Human Pose Estimation In The Wild](https://mlsanity.com/paper/1802.00434/) (2018)
- [Convolutional Pose Machines](https://mlsanity.com/paper/1602.00134/) (2016)
- [Learning Category-Specific Mesh Reconstruction from Image Collections](https://mlsanity.com/paper/1803.07549/) (2018)
- [3D Menagerie: Modeling the 3D shape and pose of animals](https://mlsanity.com/paper/1611.07700/) (2016)
- [Convolutional Mesh Regression for Single-Image Human Shape Reconstruction](https://mlsanity.com/paper/1905.03244/) (2019)
- [Learning Correspondence from the Cycle-Consistency of Time](https://mlsanity.com/paper/1903.07593/) (2019)
- [Part-based R-CNNs for Fine-grained Category Detection](https://mlsanity.com/paper/1407.3867/) (2014)
- [Three-D Safari: Learning to Estimate Zebra Pose, Shape, and Texture from Images "In the Wild"](https://mlsanity.com/paper/1908.07201/) (2019)
- [Learning Dense Correspondence via 3D-guided Cycle Consistency](https://mlsanity.com/paper/1604.05383/) (2016)
- [Creatures great and SMAL: Recovering the shape and motion of animals from video](https://mlsanity.com/paper/1811.05804/) (2018)
- [Articulation-aware Canonical Surface Mapping](https://mlsanity.com/paper/2004.00614/) (2020)
- [Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization in the Loop](https://mlsanity.com/paper/2007.11110/) (2020)
- [Continuous Surface Embeddings](https://mlsanity.com/paper/2011.12438/) (2020)
- [Deep Shells: Unsupervised Shape Correspondence with Optimal Transport](https://mlsanity.com/paper/2010.15261/) (2020)
- [Unsupervised learning of object landmarks by factorized spatial embeddings](https://mlsanity.com/paper/1705.02193/) (2017)
- [Canonical Surface Mapping via Geometric Cycle Consistency](https://mlsanity.com/paper/1907.10043/) (2019)
- [C3DPO: Canonical 3D Pose Networks for Non-Rigid Structure From Motion](https://mlsanity.com/paper/1909.02533/) (2019)
- [Parsing R-CNN for Instance-Level Human Analysis](https://mlsanity.com/paper/1811.12596/) (2018)
- [Multi-Image Matching via Fast Alternating Minimization](https://mlsanity.com/paper/1505.04845/) (2015)
- [ZoomOut: Spectral Upsampling for Efficient Shape Correspondence](https://mlsanity.com/paper/1904.07865/) (2019)
- [Transferring Dense Pose to Proximal Animal Classes](https://mlsanity.com/paper/2003.00080/) (2020)
- [WarpNet: Weakly Supervised Matching for Single-view Reconstruction](https://mlsanity.com/paper/1604.05592/) (2016)
- [Slim DensePose: Thrifty Learning from Sparse Annotations and Motion Cues](https://mlsanity.com/paper/1906.05706/) (2019)

## Built on by (most cited first, 13 of 13)

- [BANMo: Building Animatable 3D Neural Models from Many Casual Videos](https://mlsanity.com/paper/2112.12761/) (2021)
- [Animal Kingdom: A Large and Diverse Dataset for Animal Behavior Understanding](https://mlsanity.com/paper/2204.08129/) (2022)
- [Reconstructing Animatable Categories from Videos](https://mlsanity.com/paper/2305.06351/) (2023)
- [MoDA: Modeling Deformable 3D Objects from Casual Videos](https://mlsanity.com/paper/2304.08279/) (2023)
- [SHIC: Shape-Image Correspondences with no Keypoint Supervision](https://mlsanity.com/paper/2407.18907/) (2024)
- [BodyMap: Learning Full-Body Dense Correspondence Map](https://mlsanity.com/paper/2205.09111/) (2022)
- [GaussiGAN: Controllable Image Synthesis with 3D Gaussians from Unposed Silhouettes](https://mlsanity.com/paper/2106.13215/) (2021)
- [Normal-guided Garment UV Prediction for Human Re-texturing](https://mlsanity.com/paper/2303.06504/) (2023)
- [4DPV: 4D Pet from Videos by Coarse-to-Fine Non-Rigid Radiance Fields](https://mlsanity.com/paper/2411.10275/) (2024)
- [PartCo: Part-Level Correspondence Priors Enhance Category Discovery](https://mlsanity.com/paper/2509.22769/) (2025)
- [Improving Detection of Person Class Using Dense Pooling](https://mlsanity.com/paper/2410.20966/) (2024)
- [DINeMo: Learning Neural Mesh Models with no 3D Annotations](https://mlsanity.com/paper/2503.20220/) (2025)
- [Best Segmentation Buddies for Image-Shape Correspondence](https://mlsanity.com/paper/2605.18193/) (2026)

## Abstract (arXiv)

We tackle the problem of learning the geometry of multiple categories of deformable objects jointly. Recent work has shown that it is possible to learn a unified dense pose predictor for several categories of related objects. However, training such models requires to initialize inter-category correspondences by hand. This is suboptimal and the resulting models fail to maintain correct correspondences as individual categories are learned. In this paper, we show that improved correspondences can be learned automatically as a natural byproduct of learning category-specific dense pose predictors. To do this, we express correspondences between different categories and between images and categories using a unified embedding. Then, we use the latter to enforce two constraints: symmetric inter-category cycle consistency and a new asymmetric image-to-category cycle consistency. Without any manual annotations for the inter-category correspondences, we obtain state-of-the-art alignment results, outperforming dedicated methods for matching 3D shapes. Moreover, the new model is also better at the task of dense pose prediction than prior work.