# LMPNet for Weakly-supervised Keypoint Discovery

arXiv: https://arxiv.org/abs/2507.02308 · HTML: https://mlsanity.com/paper/2507.02308/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 45 references that are in the map.
- First posted on arXiv 3 Jul 2025.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations](https://mlsanity.com/paper/1811.12359/) (2018)
- [Network Dissection: Quantifying Interpretability of Deep Visual Representations](https://mlsanity.com/paper/1704.05796/) (2017)
- [Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey](https://mlsanity.com/paper/1902.06162/) (2019)
- [Network In Network](https://mlsanity.com/paper/1312.4400/) (2013)
- [DeepPose: Human Pose Estimation via Deep Neural Networks](https://mlsanity.com/paper/1312.4659/) (2013)
- [Towards a Definition of Disentangled Representations](https://mlsanity.com/paper/1812.02230/) (2018)
- [Weakly Supervised Learning of Instance Segmentation with Inter-pixel Relations](https://mlsanity.com/paper/1904.05044/) (2019)
- [Object Detectors Emerge in Deep Scene CNNs](https://mlsanity.com/paper/1412.6856/) (2014)
- [Attention-based Dropout Layer for Weakly Supervised Object Localization](https://mlsanity.com/paper/1908.10028/) (2019)
- [Unsupervised Learning of Object Landmarks through Conditional Image Generation](https://mlsanity.com/paper/1806.07823/) (2018)
- [Part-based R-CNNs for Fine-grained Category Detection](https://mlsanity.com/paper/1407.3867/) (2014)
- [Evaluating Weakly Supervised Object Localization Methods Right](https://mlsanity.com/paper/2001.07437/) (2020)
- [Weakly Supervised Instance Segmentation using Class Peak Response](https://mlsanity.com/paper/1804.00880/) (2018)
- [Unsupervised Discovery of Object Landmarks as Structural Representations](https://mlsanity.com/paper/1804.04412/) (2018)
- [The Application of Two-level Attention Models in Deep Convolutional Neural Network for Fine-grained Image Classification](https://mlsanity.com/paper/1411.6447/) (2014)
- [Unsupervised learning of object landmarks by factorized spatial embeddings](https://mlsanity.com/paper/1705.02193/) (2017)
- [Learning Deep Context-aware Features over Body and Latent Parts for Person Re-identification](https://mlsanity.com/paper/1710.06555/) (2017)
- [Saliency Guided Self-attention Network for Weakly and Semi-supervised Semantic Segmentation](https://mlsanity.com/paper/1910.05475/) (2019)
- [On learning to localize objects with minimal supervision](https://mlsanity.com/paper/1403.1024/) (2014)
- [AnchorNet: A Weakly Supervised Network to Learn Geometry-sensitive Features For Semantic Matching](https://mlsanity.com/paper/1704.04749/) (2017)
- [Aligned to the Object, not to the Image: A Unified Pose-aligned Representation for Fine-grained Recognition](https://mlsanity.com/paper/1801.09057/) (2018)

## Abstract (arXiv)

In this work, we explore the task of semantic object keypoint discovery weakly-supervised by only category labels. This is achieved by transforming discriminatively-trained intermediate layer filters into keypoint detectors. We begin by identifying three preferred characteristics of keypoint detectors: (i) spatially sparse activations, (ii) consistency and (iii) diversity. Instead of relying on hand-crafted loss terms, a novel computationally-efficient leaky max pooling (LMP) layer is proposed to explicitly encourage final conv-layer filters to learn "non-repeatable local patterns" that are well aligned with object keypoints. Informed by visualizations, a simple yet effective selection strategy is proposed to ensure consistent filter activations and attention mask-out is then applied to force the network to distribute its attention to the whole object instead of just the most discriminative region. For the final keypoint prediction, a learnable clustering layer is proposed to group keypoint proposals into keypoint predictions. The final model, named LMPNet, is highly interpretable in that it directly manipulates network filters to detect predefined concepts. Our experiments show that LMPNet can (i) automatically discover semantic keypoints that are robust to object pose and (ii) achieves strong prediction accuracy comparable to a supervised pose estimation model.