# Multi-level Supervised Contrastive Learning

arXiv: https://arxiv.org/abs/2502.02202 · HTML: https://mlsanity.com/paper/2502.02202/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 41 references that are in the map.
- First posted on arXiv 4 Feb 2025.

## Builds on (most cited first, 30 of 30)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [An overview of gradient descent optimization algorithms](https://mlsanity.com/paper/1609.04747/) (2016)
- [Learning Representations by Maximizing Mutual Information Across Views](https://mlsanity.com/paper/1906.00910/) (2019)
- [Data-Efficient Image Recognition with Contrastive Predictive Coding](https://mlsanity.com/paper/1905.09272/) (2019)
- [Hard Negative Mixing for Contrastive Learning](https://mlsanity.com/paper/2010.01028/) (2020)
- [Understanding the Behaviour of Contrastive Loss](https://mlsanity.com/paper/2012.09740/) (2020)
- [Debiased Contrastive Learning](https://mlsanity.com/paper/2007.00224/) (2020)
- [Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning](https://mlsanity.com/paper/2011.01403/) (2020)
- [Targeted Supervised Contrastive Learning for Long-Tailed Recognition](https://mlsanity.com/paper/2111.13998/) (2021)
- [MoPro: Webly Supervised Learning with Momentum Prototypes](https://mlsanity.com/paper/2009.07995/) (2020)
- [Use All The Labels: A Hierarchical Multi-Label Contrastive Learning Framework](https://mlsanity.com/paper/2204.13207/) (2022)
- [Investigating Why Contrastive Learning Benefits Robustness Against Label Noise](https://mlsanity.com/paper/2201.12498/) (2022)
- [Perfectly Balanced: Improving Transfer and Robustness of Supervised Contrastive Learning](https://mlsanity.com/paper/2204.07596/) (2022)
- [Unbiased Supervised Contrastive Learning](https://mlsanity.com/paper/2211.05568/) (2022)
- [Contrastive Learning Improves Model Robustness Under Label Noise](https://mlsanity.com/paper/2104.08984/) (2021)
- [Multi-Label Contrastive Learning for Abstract Visual Reasoning](https://mlsanity.com/paper/2012.01944/) (2020)
- [On Learning Contrastive Representations for Learning with Noisy Labels](https://mlsanity.com/paper/2203.01785/) (2022)
- [Leveraging Class Hierarchies with Metric-Guided Prototype Learning](https://mlsanity.com/paper/2007.03047/) (2020)
- [Multi-Label Image Classification with Contrastive Learning](https://mlsanity.com/paper/2107.11626/) (2021)
- [End-to-End Supervised Multilabel Contrastive Learning](https://mlsanity.com/paper/2307.03967/) (2023)
- [Adaptive Multi-head Contrastive Learning](https://mlsanity.com/paper/2310.05615/) (2023)
- [A Framework using Contrastive Learning for Classification with Noisy Labels](https://mlsanity.com/paper/2104.09563/) (2021)

## Built on by (most cited first, 1 of 1)

- [Beyond Flat Labels: Level-Restricted Contrastive Learning for Hierarchical Fine-Grained Vision Classification](https://mlsanity.com/paper/2606.21838/) (2026)

## Abstract (arXiv)

Contrastive learning is a well-established paradigm in representation learning. The standard framework of contrastive learning minimizes the distance between "similar" instances and maximizes the distance between dissimilar ones in the projection space, disregarding the various aspects of similarity that can exist between two samples. Current methods rely on a single projection head, which fails to capture the full complexity of different aspects of a sample, leading to suboptimal performance, especially in scenarios with limited training data. In this paper, we present a novel supervised contrastive learning method in a unified framework called multilevel contrastive learning (MLCL), that can be applied to both multi-label and hierarchical classification tasks. The key strength of the proposed method is the ability to capture similarities between samples across different labels and/or hierarchies using multiple projection heads. Extensive experiments on text and image datasets demonstrate that the proposed approach outperforms state-of-the-art contrastive learning methods