# Continual Self-supervised Learning Considering Medical Domain Knowledge in Chest CT Images

arXiv: https://arxiv.org/abs/2501.04217 · HTML: https://mlsanity.com/paper/2501.04217/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 26 references that are in the map.
- First posted on arXiv 8 Jan 2025.

## Builds on (most cited first, 18 of 18)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Dark Experience for General Continual Learning: a Strong, Simple Baseline](https://mlsanity.com/paper/2004.07211/) (2020)
- [Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey](https://mlsanity.com/paper/1902.06162/) (2019)
- [Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference](https://mlsanity.com/paper/1810.11910/) (2018)
- [3D Self-Supervised Methods for Medical Imaging](https://mlsanity.com/paper/2006.03829/) (2020)
- [Self-supervised learning methods and applications in medical imaging analysis: A survey](https://mlsanity.com/paper/2109.08685/) (2021)
- [Multimodal Self-Supervised Learning for Medical Image Analysis](https://mlsanity.com/paper/1912.05396/) (2019)
- [UniMiSS: Universal Medical Self-Supervised Learning via Breaking Dimensionality Barrier](https://mlsanity.com/paper/2112.09356/) (2021)
- [Continual Self-supervised Learning: Towards Universal Multi-modal Medical Data Representation Learning](https://mlsanity.com/paper/2311.17597/) (2023)
- [Anatomical Invariance Modeling and Semantic Alignment for Self-supervised Learning in 3D Medical Image Analysis](https://mlsanity.com/paper/2302.05615/) (2023)
- [Hierarchical Self-Supervised Learning for Medical Image Segmentation Based on Multi-Domain Data Aggregation](https://mlsanity.com/paper/2107.04886/) (2021)
- [Contrastive Semi-supervised Learning for Domain Adaptive Segmentation Across Similar Anatomical Structures](https://mlsanity.com/paper/2208.08605/) (2022)
- [Self-Knowledge Distillation based Self-Supervised Learning for Covid-19 Detection from Chest X-Ray Images](https://mlsanity.com/paper/2206.03009/) (2022)
- [Self-Supervised Learning for Gastritis Detection with Gastric X-ray Images](https://mlsanity.com/paper/2104.02864/) (2021)
- [COVID-19 Detection Based on Self-Supervised Transfer Learning Using Chest X-Ray Images](https://mlsanity.com/paper/2212.09276/) (2022)
- [Boosting Automatic COVID-19 Detection Performance with Self-Supervised Learning and Batch Knowledge Ensembling](https://mlsanity.com/paper/2212.09281/) (2022)
- [Dual Windows Are Significant: Learning from Mediastinal Window and Focusing on Lung Window](https://mlsanity.com/paper/2206.03803/) (2022)

## Built on by (most cited first, 5 of 5)

- [Continual Retinal Vision-Language Pre-training upon Incremental Imaging Modalities](https://mlsanity.com/paper/2506.19320/) (2025)
- [Privacy-Aware Continual Self-Supervised Learning on Multi-Window Chest Computed Tomography for Domain-Shift Robustness](https://mlsanity.com/paper/2510.27213/) (2025)
- [Forging a Dynamic Memory: Retrieval-Guided Continual Learning for Generalist Medical Foundation Models](https://mlsanity.com/paper/2512.13072/) (2025)
- [InvCoSS: Inversion-driven Continual Self-supervised Learning in Medical Multi-modal Image Pre-training](https://mlsanity.com/paper/2512.19213/) (2025)
- [Dual-Model Weight Selection and Self-Knowledge Distillation for Medical Image Classification](https://mlsanity.com/paper/2508.20461/) (2025)

## Abstract (arXiv)

We propose a novel continual self-supervised learning method (CSSL) considering medical domain knowledge in chest CT images. Our approach addresses the challenge of sequential learning by effectively capturing the relationship between previously learned knowledge and new information at different stages. By incorporating an enhanced DER into CSSL and maintaining both diversity and representativeness within the rehearsal buffer of DER, the risk of data interference during pretraining is reduced, enabling the model to learn more richer and robust feature representations. In addition, we incorporate a mixup strategy and feature distillation to further enhance the model's ability to learn meaningful representations. We validate our method using chest CT images obtained under two different imaging conditions, demonstrating superior performance compared to state-of-the-art methods.