# Representation Learning with Information Theory for COVID-19 Detection

arXiv: https://arxiv.org/abs/2207.01437 · HTML: https://mlsanity.com/paper/2207.01437/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 40 references that are in the map.
- First posted on arXiv 4 Jul 2022.

## Builds on (most cited first, 14 of 14)

- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results](https://mlsanity.com/paper/1703.01780/) (2017)
- [Learning Discrete Representations via Information Maximizing Self-Augmented Training](https://mlsanity.com/paper/1702.08720/) (2017)
- [Transformation Consistent Self-ensembling Model for Semi-supervised Medical Image Segmentation](https://mlsanity.com/paper/1903.00348/) (2019)
- [Deep Transparent Prediction through Latent Representation Analysis](https://mlsanity.com/paper/2009.07044/) (2020)
- [MIA-COV19D: COVID-19 Detection through 3-D Chest CT Image Analysis](https://mlsanity.com/paper/2106.07524/) (2021)
- [Semi-Supervised Medical Image Segmentation via Learning Consistency under Transformations](https://mlsanity.com/paper/1911.01218/) (2019)
- [Semi-supervised Medical Image Classification with Relation-driven Self-ensembling Model](https://mlsanity.com/paper/2005.07377/) (2020)
- [Semi-Supervised Brain Lesion Segmentation with an Adapted Mean Teacher Model](https://mlsanity.com/paper/1903.01248/) (2019)
- [Consistency Regularization for Variational Auto-Encoders](https://mlsanity.com/paper/2105.14859/) (2021)
- [AI-MIA: COVID-19 Detection & Severity Analysis through Medical Imaging](https://mlsanity.com/paper/2206.04732/) (2022)
- [Boosting Semi-supervised Image Segmentation with Global and Local Mutual Information Regularization](https://mlsanity.com/paper/2103.04813/) (2021)

## Built on by (most cited first, 1 of 1)

- [When More is Less: Incorporating Additional Datasets Can Hurt Performance By Introducing Spurious Correlations](https://mlsanity.com/paper/2308.04431/) (2023)

## Abstract (arXiv)

Successful data representation is a fundamental factor in machine learning based medical imaging analysis. Deep Learning (DL) has taken an essential role in robust representation learning. However, the inability of deep models to generalize to unseen data can quickly overfit intricate patterns. Thereby, we can conveniently implement strategies to aid deep models in discovering useful priors from data to learn their intrinsic properties. Our model, which we call a dual role network (DRN), uses a dependency maximization approach based on Least Squared Mutual Information (LSMI). The LSMI leverages dependency measures to ensure representation invariance and local smoothness. While prior works have used information theory measures like mutual information, known to be computationally expensive due to a density estimation step, our LSMI formulation alleviates the issues of intractable mutual information estimation and can be used to approximate it. Experiments on CT based COVID-19 Detection and COVID-19 Severity Detection benchmarks demonstrate the effectiveness of our method.