# Weakly Supervised Keypoint Discovery

arXiv: https://arxiv.org/abs/2109.13423 · HTML: https://mlsanity.com/paper/2109.13423/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 50 references that are in the map.
- First posted on arXiv 28 Sep 2021.

## Builds on (most cited first, 30 of 30)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [Cascaded Pyramid Network for Multi-Person Pose Estimation](https://mlsanity.com/paper/1711.07319/) (2017)
- [Convolutional Pose Machines](https://mlsanity.com/paper/1602.00134/) (2016)
- [Hide-and-Seek: Forcing a Network to be Meticulous for Weakly-supervised Object and Action Localization](https://mlsanity.com/paper/1704.04232/) (2017)
- [Markerless tracking of user-defined features with deep learning](https://mlsanity.com/paper/1804.03142/) (2018)
- [Dynamic Multiscale Graph Neural Networks for 3D Skeleton-Based Human Motion Prediction](https://mlsanity.com/paper/2003.08802/) (2020)
- [Unsupervised Learning of Object Landmarks through Conditional Image Generation](https://mlsanity.com/paper/1806.07823/) (2018)
- [Cross-Domain Adaptation for Animal Pose Estimation](https://mlsanity.com/paper/1908.05806/) (2019)
- [Constrained Convolutional Neural Networks for Weakly Supervised Segmentation](https://mlsanity.com/paper/1506.03648/) (2015)
- [PCL: Proposal Cluster Learning for Weakly Supervised Object Detection](https://mlsanity.com/paper/1807.03342/) (2018)
- [Pose-driven Deep Convolutional Model for Person Re-identification](https://mlsanity.com/paper/1709.08325/) (2017)
- [Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization in the Loop](https://mlsanity.com/paper/2007.11110/) (2020)
- [Unsupervised Discovery of Object Landmarks as Structural Representations](https://mlsanity.com/paper/1804.04412/) (2018)
- [Unsupervised Part-Based Disentangling of Object Shape and Appearance](https://mlsanity.com/paper/1903.06946/) (2019)
- [Unsupervised learning of object landmarks by factorized spatial embeddings](https://mlsanity.com/paper/1705.02193/) (2017)
- [Multi-Attention Multi-Class Constraint for Fine-grained Image Recognition](https://mlsanity.com/paper/1806.05372/) (2018)
- [Bird Species Categorization Using Pose Normalized Deep Convolutional Nets](https://mlsanity.com/paper/1406.2952/) (2014)
- [Deep Self-Taught Learning for Weakly Supervised Object Localization](https://mlsanity.com/paper/1704.05188/) (2017)
- [Anchor Loss: Modulating Loss Scale based on Prediction Difficulty](https://mlsanity.com/paper/1909.11155/) (2019)
- [Articulated motion discovery using pairs of trajectories](https://mlsanity.com/paper/1411.7883/) (2014)
- [AnchorNet: A Weakly Supervised Network to Learn Geometry-sensitive Features For Semantic Matching](https://mlsanity.com/paper/1704.04749/) (2017)
- [Comparator Networks](https://mlsanity.com/paper/1807.11440/) (2018)
- [Aligned to the Object, not to the Image: A Unified Pose-aligned Representation for Fine-grained Recognition](https://mlsanity.com/paper/1801.09057/) (2018)
- [Unsupervised Part Mining for Fine-grained Image Classification](https://mlsanity.com/paper/1902.09941/) (2019)

## Built on by (most cited first, 5 of 5)

- [Self-Supervised Keypoint Discovery in Behavioral Videos](https://mlsanity.com/paper/2112.05121/) (2021)
- [Demystifying Unsupervised Semantic Correspondence Estimation](https://mlsanity.com/paper/2207.05054/) (2022)
- [BKinD-3D: Self-Supervised 3D Keypoint Discovery from Multi-View Videos](https://mlsanity.com/paper/2212.07401/) (2022)
- [Learning Keypoints for Multi-Agent Behavior Analysis using Self-Supervision](https://mlsanity.com/paper/2409.09455/) (2024)
- [SAVeD: Learning to Denoise Low-SNR Video for Improved Downstream Performance](https://mlsanity.com/paper/2504.00161/) (2025)

## Abstract (arXiv)

In this paper, we propose a method for keypoint discovery from a 2D image using image-level supervision. Recent works on unsupervised keypoint discovery reliably discover keypoints of aligned instances. However, when the target instances have high viewpoint or appearance variation, the discovered keypoints do not match the semantic correspondences over different images. Our work aims to discover keypoints even when the target instances have high viewpoint and appearance variation by using image-level supervision. Motivated by the weakly-supervised learning approach, our method exploits image-level supervision to identify discriminative parts and infer the viewpoint of the target instance. To discover diverse parts, we adopt a conditional image generation approach using a pair of images with structural deformation. Finally, we enforce a viewpoint-based equivariance constraint using the keypoints from the image-level supervision to resolve the spatial correlation problem that consistently appears in the images taken from various viewpoints. Our approach achieves state-of-the-art performance for the task of keypoint estimation on the limited supervision scenarios. Furthermore, the discovered keypoints are directly applicable to downstream tasks without requiring any keypoint labels.