# The Intrinsic Manifolds of Radiological Images and their Role in Deep Learning

arXiv: https://arxiv.org/abs/2207.02797 · HTML: https://mlsanity.com/paper/2207.02797/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 33 references that are in the map.
- First posted on arXiv 6 Jul 2022.

## Builds on (most cited first, 16 of 16)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size](https://mlsanity.com/paper/1602.07360/) (2016)
- [A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets](https://mlsanity.com/paper/1707.08819/) (2017)
- [The Intrinsic Dimension of Images and Its Impact on Learning](https://mlsanity.com/paper/2104.08894/) (2021)
- [Intrinsic dimension of data representations in deep neural networks](https://mlsanity.com/paper/1905.12784/) (2019)
- [Estimating the intrinsic dimension of datasets by a minimal neighborhood information](https://mlsanity.com/paper/1803.06992/) (2018)
- [Intrinsic Dimension, Persistent Homology and Generalization in Neural Networks](https://mlsanity.com/paper/2111.13171/) (2021)
- [Scikit-dimension: a Python package for intrinsic dimension estimation](https://mlsanity.com/paper/2109.02596/) (2021)
- [Automatic Knee Osteoarthritis Diagnosis from Plain Radiographs: A Deep Learning-Based Approach](https://mlsanity.com/paper/1710.10589/) (2017)
- [Automatic Detection of Knee Joints and Quantification of Knee Osteoarthritis Severity using Convolutional Neural Networks](https://mlsanity.com/paper/1703.09856/) (2017)
- [Geometry-Aware Maximum Likelihood Estimation of Intrinsic Dimension](https://mlsanity.com/paper/1904.06151/) (2019)
- [Bridging the gap between Natural and Medical Images through Deep Colorization](https://mlsanity.com/paper/2005.10589/) (2020)

## Built on by (most cited first, 10 of 10)

- [The Effect of Intrinsic Dataset Properties on Generalization: Unraveling Learning Differences Between Natural and Medical Images](https://mlsanity.com/paper/2401.08865/) (2024)
- [Reverse Engineering Breast MRIs: Predicting Acquisition Parameters Directly from Images](https://mlsanity.com/paper/2303.04911/) (2023)
- [Evaluating Visual Explanations of Attention Maps for Transformer-based Medical Imaging](https://mlsanity.com/paper/2503.09535/) (2025)
- [MiSuRe is all you need to explain your image segmentation](https://mlsanity.com/paper/2406.12173/) (2024)
- [Exploring Patient Data Requirements in Training Effective AI Models for MRI-based Breast Cancer Classification](https://mlsanity.com/paper/2502.18506/) (2025)
- [Pre-processing and Compression: Understanding Hidden Representation Refinement Across Imaging Domains via Intrinsic Dimension](https://mlsanity.com/paper/2408.08381/) (2024)
- [CheXLearner: Text-Guided Fine-Grained Representation Learning for Progression Detection](https://mlsanity.com/paper/2505.06903/) (2025)
- [Density Estimation via Binless Multidimensional Integration](https://mlsanity.com/paper/2407.08094/) (2024)
- [Towards Affordable Tumor Segmentation and Visualization for 3D Breast MRI Using SAM2](https://mlsanity.com/paper/2507.23272/) (2025)
- [Spectral Consistent Flow for One-step 3D Medical Image Translation](https://mlsanity.com/paper/2607.10627/) (2026)

## Abstract (arXiv)

The manifold hypothesis is a core mechanism behind the success of deep learning, so understanding the intrinsic manifold structure of image data is central to studying how neural networks learn from the data. Intrinsic dataset manifolds and their relationship to learning difficulty have recently begun to be studied for the common domain of natural images, but little such research has been attempted for radiological images. We address this here. First, we compare the intrinsic manifold dimensionality of radiological and natural images. We also investigate the relationship between intrinsic dimensionality and generalization ability over a wide range of datasets. Our analysis shows that natural image datasets generally have a higher number of intrinsic dimensions than radiological images. However, the relationship between generalization ability and intrinsic dimensionality is much stronger for medical images, which could be explained as radiological images having intrinsic features that are more difficult to learn. These results give a more principled underpinning for the intuition that radiological images can be more challenging to apply deep learning to than natural image datasets common to machine learning research. We believe rather than directly applying models developed for natural images to the radiological imaging domain, more care should be taken to developing architectures and algorithms that are more tailored to the specific characteristics of this domain. The research shown in our paper, demonstrating these characteristics and the differences from natural images, is an important first step in this direction.