# CoBooM: Codebook Guided Bootstrapping for Medical Image Representation Learning

arXiv: https://arxiv.org/abs/2408.04262 · HTML: https://mlsanity.com/paper/2408.04262/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 34 references that are in the map.
- First posted on arXiv 8 Aug 2024.

## Builds on (most cited first, 27 of 27)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning](https://mlsanity.com/paper/2105.04906/) (2021)
- [ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases](https://mlsanity.com/paper/1705.02315/) (2017)
- [Large Batch Training of Convolutional Networks](https://mlsanity.com/paper/1708.03888/) (2017)
- [Benchmarking Self-Supervised Learning on Diverse Pathology Datasets](https://mlsanity.com/paper/2212.04690/) (2022)
- [Self supervised contrastive learning for digital histopathology](https://mlsanity.com/paper/2011.13971/) (2020)
- [MoCo-CXR: MoCo Pretraining Improves Representation and Transferability of Chest X-ray Models](https://mlsanity.com/paper/2010.05352/) (2020)
- [Preservational Learning Improves Self-supervised Medical Image Models by Reconstructing Diverse Contexts](https://mlsanity.com/paper/2109.04379/) (2021)
- [Online Clustered Codebook](https://mlsanity.com/paper/2307.15139/) (2023)
- [MedAug: Contrastive learning leveraging patient metadata improves representations for chest X-ray interpretation](https://mlsanity.com/paper/2102.10663/) (2021)
- [A Benchmark for Studying Diabetic Retinopathy: Segmentation, Grading, and Transferability](https://mlsanity.com/paper/2008.09772/) (2020)
- [DiRA: Discriminative, Restorative, and Adversarial Learning for Self-supervised Medical Image Analysis](https://mlsanity.com/paper/2204.10437/) (2022)
- [PCRLv2: A Unified Visual Information Preservation Framework for Self-supervised Pre-training in Medical Image Analysis](https://mlsanity.com/paper/2301.00772/) (2023)
- [Self-Supervised Representation Learning using Visual Field Expansion on Digital Pathology](https://mlsanity.com/paper/2109.03299/) (2021)
- [Intermediate Layers Matter in Momentum Contrastive Self Supervised Learning](https://mlsanity.com/paper/2110.14805/) (2021)
- [Multi-Label Retinal Disease Classification using Transformers](https://mlsanity.com/paper/2207.02335/) (2022)
- [CAiD: Context-Aware Instance Discrimination for Self-supervised Learning in Medical Imaging](https://mlsanity.com/paper/2204.07344/) (2022)
- [Contrastive Quantization with Code Memory for Unsupervised Image Retrieval](https://mlsanity.com/paper/2109.05205/) (2021)
- [SynergyNet: Bridging the Gap between Discrete and Continuous Representations for Precise Medical Image Segmentation](https://mlsanity.com/paper/2310.17764/) (2023)
- [Decomposing Normal and Abnormal Features of Medical Images for Content-based Image Retrieval](https://mlsanity.com/paper/2011.06224/) (2020)

## Built on by (most cited first, 2 of 2)

- [RobSurv: Vector Quantization-Based Multi-Modal Learning for Robust Cancer Survival Prediction](https://mlsanity.com/paper/2505.02529/) (2025)
- [DiSSECT: Structuring Transfer-Ready Medical Image Representations through Discrete Self-Supervision](https://mlsanity.com/paper/2509.18765/) (2025)

## Abstract (arXiv)

Self-supervised learning (SSL) has emerged as a promising paradigm for medical image analysis by harnessing unannotated data. Despite their potential, the existing SSL approaches overlook the high anatomical similarity inherent in medical images. This makes it challenging for SSL methods to capture diverse semantic content in medical images consistently. This work introduces a novel and generalized solution that implicitly exploits anatomical similarities by integrating codebooks in SSL. The codebook serves as a concise and informative dictionary of visual patterns, which not only aids in capturing nuanced anatomical details but also facilitates the creation of robust and generalized feature representations. In this context, we propose CoBooM, a novel framework for self-supervised medical image learning by integrating continuous and discrete representations. The continuous component ensures the preservation of fine-grained details, while the discrete aspect facilitates coarse-grained feature extraction through the structured embedding space. To understand the effectiveness of CoBooM, we conduct a comprehensive evaluation of various medical datasets encompassing chest X-rays and fundus images. The experimental results reveal a significant performance gain in classification and segmentation tasks.