# A novel multiple instance learning framework for COVID-19 severity assessment via data augmentation and self-supervised learning

arXiv: https://arxiv.org/abs/2102.03837 · HTML: https://mlsanity.com/paper/2102.03837/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 47 references that are in the map.
- First posted on arXiv 7 Feb 2021.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [Random Erasing Data Augmentation](https://mlsanity.com/paper/1708.04896/) (2017)
- [Colorful Image Colorization](https://mlsanity.com/paper/1603.08511/) (2016)
- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [Unsupervised Visual Representation Learning by Context Prediction](https://mlsanity.com/paper/1505.05192/) (2015)
- [The Effectiveness of Data Augmentation in Image Classification using Deep Learning](https://mlsanity.com/paper/1712.04621/) (2017)
- [Learning Representations for Automatic Colorization](https://mlsanity.com/paper/1603.06668/) (2016)
- [Not-so-supervised: a survey of semi-supervised, multi-instance, and transfer learning in medical image analysis](https://mlsanity.com/paper/1804.06353/) (2018)
- [Patch-based Convolutional Neural Network for Whole Slide Tissue Image Classification](https://mlsanity.com/paper/1504.07947/) (2015)
- [Boosting Few-Shot Visual Learning with Self-Supervision](https://mlsanity.com/paper/1906.05186/) (2019)
- [Synthetic Data Augmentation using GAN for Improved Liver Lesion Classification](https://mlsanity.com/paper/1801.02385/) (2018)
- [Smart Augmentation - Learning an Optimal Data Augmentation Strategy](https://mlsanity.com/paper/1703.08383/) (2017)
- [Exploiting the potential of unlabeled endoscopic video data with self-supervised learning](https://mlsanity.com/paper/1711.09726/) (2017)
- [Severity Assessment of Coronavirus Disease 2019 (COVID-19) Using Quantitative Features from Chest CT Images](https://mlsanity.com/paper/2003.11988/) (2020)
- [Diversity Helps: Unsupervised Few-shot Learning via Distribution Shift-based Data Augmentation](https://mlsanity.com/paper/2004.05805/) (2020)
- [Multiple Instance Learning Convolutional Neural Networks for Object Recognition](https://mlsanity.com/paper/1610.03155/) (2016)
- [Semi-supervised Learning using Denoising Autoencoders for Brain Lesion Detection and Segmentation](https://mlsanity.com/paper/1611.08664/) (2016)

## Built on by (most cited first, 19 of 19)

- [Multiple Instance Learning for Digital Pathology: A Review on the State-of-the-Art, Limitations & Future Potential](https://mlsanity.com/paper/2206.04425/) (2022)
- [Anomaly Detection-Inspired Few-Shot Medical Image Segmentation Through Self-Supervision With Supervoxels](https://mlsanity.com/paper/2203.02048/) (2022)
- [Label-Efficient Deep Learning in Medical Image Analysis: Challenges and Future Directions](https://mlsanity.com/paper/2303.12484/) (2023)
- [A Review of Predictive and Contrastive Self-supervised Learning for Medical Images](https://mlsanity.com/paper/2302.05043/) (2023)
- [Better Pseudo-label: Joint Domain-aware Label and Dual-classifier for Semi-supervised Domain Generalization](https://mlsanity.com/paper/2110.04820/) (2021)
- [MixUp-MIL: Novel Data Augmentation for Multiple Instance Learning and a Study on Thyroid Cancer Diagnosis](https://mlsanity.com/paper/2211.05862/) (2022)
- [Temporal Context Matters: Enhancing Single Image Prediction with Disease Progression Representations](https://mlsanity.com/paper/2203.01933/) (2022)
- [MixBag: Bag-Level Data Augmentation for Learning from Label Proportions](https://mlsanity.com/paper/2308.08822/) (2023)
- [Data efficient deep learning for medical image analysis: A survey](https://mlsanity.com/paper/2310.06557/) (2023)
- [Attention2Minority: A salient instance inference-based multiple instance learning for classifying small lesions in whole slide images](https://mlsanity.com/paper/2301.07700/) (2023)
- [Information Bottleneck Attribution for Visual Explanations of Diagnosis and Prognosis](https://mlsanity.com/paper/2104.02869/) (2021)
- [Lung Swapping Autoencoder: Learning a Disentangled Structure-texture Representation of Chest Radiographs](https://mlsanity.com/paper/2201.07344/) (2022)
- [Learning Self-Supervised Representations for Label Efficient Cross-Domain Knowledge Transfer on Diabetic Retinopathy Fundus Images](https://mlsanity.com/paper/2304.11168/) (2023)
- [Learning from Majority Label: A Novel Problem in Multi-class Multiple-Instance Learning](https://mlsanity.com/paper/2509.04023/) (2025)
- [MixUp-MIL: A Study on Linear & Multilinear Interpolation-Based Data Augmentation for Whole Slide Image Classification](https://mlsanity.com/paper/2311.03052/) (2023)
- [Dive into Self-Supervised Learning for Medical Image Analysis: Data, Models and Tasks](https://mlsanity.com/paper/2209.12157/) (2022)
- [Multi-instance Learning as Downstream Task of Self-Supervised Learning-based Pre-trained Model](https://mlsanity.com/paper/2505.21564/) (2025)
- [Knowing When Not to Predict: Self Supervised Learning and Abstention for Safer DR Screening](https://mlsanity.com/paper/2605.19133/) (2026)
- [Towards Whole-Study Screening for Congenital Heart Disease in Fetal Ultrasound Using Multiple Instance Learning](https://mlsanity.com/paper/2609.31376/) (2026)

## Abstract (arXiv)

How to fast and accurately assess the severity level of COVID-19 is an essential problem, when millions of people are suffering from the pandemic around the world. Currently, the chest CT is regarded as a popular and informative imaging tool for COVID-19 diagnosis. However, we observe that there are two issues -- weak annotation and insufficient data that may obstruct automatic COVID-19 severity assessment with CT images. To address these challenges, we propose a novel three-component method, i.e., 1) a deep multiple instance learning component with instance-level attention to jointly classify the bag and also weigh the instances, 2) a bag-level data augmentation component to generate virtual bags by reorganizing high confidential instances, and 3) a self-supervised pretext component to aid the learning process. We have systematically evaluated our method on the CT images of 229 COVID-19 cases, including 50 severe and 179 non-severe cases. Our method could obtain an average accuracy of 95.8%, with 93.6% sensitivity and 96.4% specificity, which outperformed previous works.