# ConPro: Learning Severity Representation for Medical Images using Contrastive Learning and Preference Optimization

arXiv: https://arxiv.org/abs/2404.18831 · HTML: https://mlsanity.com/paper/2404.18831/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 45 references that are in the map.
- First posted on arXiv 29 Apr 2024.

## Builds on (most cited first, 23 of 23)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [SimMIM: A Simple Framework for Masked Image Modeling](https://mlsanity.com/paper/2111.09886/) (2021)
- [What Makes for Good Views for Contrastive Learning?](https://mlsanity.com/paper/2005.10243/) (2020)
- [Data-Efficient Image Recognition with Contrastive Predictive Coding](https://mlsanity.com/paper/1905.09272/) (2019)
- [S4L: Self-Supervised Semi-Supervised Learning](https://mlsanity.com/paper/1905.03670/) (2019)
- [VinDr-Mammo: A large-scale benchmark dataset for computer-aided diagnosis in full-field digital mammography](https://mlsanity.com/paper/2203.11205/) (2022)
- [Contrastive Explanations for Model Interpretability](https://mlsanity.com/paper/2103.01378/) (2021)
- [Contrastive Explanation: A Structural-Model Approach](https://mlsanity.com/paper/1811.03163/) (2018)
- [Robust and Efficient Medical Imaging with Self-Supervision](https://mlsanity.com/paper/2205.09723/) (2022)
- [Max-Margin Contrastive Learning](https://mlsanity.com/paper/2112.11450/) (2021)
- [Consistent Explanations by Contrastive Learning](https://mlsanity.com/paper/2110.00527/) (2021)
- [The Effect of Intrinsic Dataset Properties on Generalization: Unraveling Learning Differences Between Natural and Medical Images](https://mlsanity.com/paper/2401.08865/) (2024)
- [Detection and severity classification of COVID-19 in CT images using deep learning](https://mlsanity.com/paper/2102.07726/) (2021)
- [Bridging the gap between Natural and Medical Images through Deep Colorization](https://mlsanity.com/paper/2005.10589/) (2020)
- [Explanation Guided Contrastive Learning for Sequential Recommendation](https://mlsanity.com/paper/2209.01347/) (2022)
- [Contrastive Corpus Attribution for Explaining Representations](https://mlsanity.com/paper/2210.00107/) (2022)
- [Plackett-Luce model for learning-to-rank task](https://mlsanity.com/paper/1909.06722/) (2019)
- [A Novel Transparency Strategy-based Data Augmentation Approach for BI-RADS Classification of Mammograms](https://mlsanity.com/paper/2203.10609/) (2022)

## Built on by (most cited first, 3 of 3)

- [A Comprehensive Survey of Direct Preference Optimization: Datasets, Theories, Variants, and Applications](https://mlsanity.com/paper/2410.15595/) (2024)
- [SHeRL-FL: When Representation Learning Meets Split Learning in Hierarchical Federated Learning](https://mlsanity.com/paper/2508.08339/) (2025)
- [Learning How Much, Not Just What: Cross-Patient Burden Order for CT Vision-Language Pretraining](https://mlsanity.com/paper/2608.00231/) (2026)

## Abstract (arXiv)

Understanding the severity of conditions shown in images in medical diagnosis is crucial, serving as a key guide for clinical assessment, treatment, as well as evaluating longitudinal progression. This paper proposes Con- PrO: a novel representation learning method for severity assessment in medical images using Contrastive learningintegrated Preference Optimization. Different from conventional contrastive learning methods that maximize the distance between classes, ConPrO injects into the latent vector the distance preference knowledge between various severity classes and the normal class. We systematically examine the key components of our framework to illuminate how contrastive prediction tasks acquire valuable representations. We show that our representation learning framework offers valuable severity ordering in the feature space while outperforming previous state-of-the-art methods on classification tasks. We achieve a 6% and 20% relative improvement compared to a supervised and a self-supervised baseline, respectively. In addition, we derived discussions on severity indicators and related applications of preference comparison in the medical domain.