# Deep Learning in current Neuroimaging: a multivariate approach with power and type I error control but arguable generalization ability

arXiv: https://arxiv.org/abs/2103.16685 · HTML: https://mlsanity.com/paper/2103.16685/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 75 references that are in the map.
- First posted on arXiv 30 Mar 2021.

## Builds on (most cited first, 13 of 13)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [Exploring Generalization in Deep Learning](https://mlsanity.com/paper/1706.08947/) (2017)
- [Norm-Based Capacity Control in Neural Networks](https://mlsanity.com/paper/1503.00036/) (2015)
- [Generalization in Deep Learning](https://mlsanity.com/paper/1710.05468/) (2017)
- [Convolutional Neural Networks for Classification of Alzheimer's Disease: Overview and Reproducible Evaluation](https://mlsanity.com/paper/1904.07773/) (2019)
- [Cross-validation failure: small sample sizes lead to large error bars](https://mlsanity.com/paper/1706.07581/) (2017)
- [Deep Learning in Alzheimer's disease: Diagnostic Classification and Prognostic Prediction using Neuroimaging Data](https://mlsanity.com/paper/1905.00931/) (2019)
- [Can parametric statistical methods be trusted for fMRI based group studies?](https://mlsanity.com/paper/1511.01863/) (2015)
- [Predicting Alzheimer's disease: a neuroimaging study with 3D convolutional neural networks](https://mlsanity.com/paper/1502.02506/) (2015)
- [Better-Than-Chance Classification for Signal Detection](https://mlsanity.com/paper/1608.08873/) (2016)
- [Statistical Agnostic Mapping: a Framework in Neuroimaging based on Concentration Inequalities](https://mlsanity.com/paper/1912.12274/) (2019)

## Built on by (most cited first, 4 of 4)

- [Automatic Diagnosis of Schizophrenia in EEG Signals Using CNN-LSTM Models](https://mlsanity.com/paper/2109.01120/) (2021)
- [Applications of Deep Learning Techniques for Automated Multiple Sclerosis Detection Using Magnetic Resonance Imaging: A Review](https://mlsanity.com/paper/2105.04881/) (2021)
- [Patchwork Learning: A Paradigm Towards Integrative Analysis across Diverse Biomedical Data Sources](https://mlsanity.com/paper/2305.06217/) (2023)
- [Automatic Diagnosis of COVID-19 from CT Images using CycleGAN and Transfer Learning](https://mlsanity.com/paper/2104.11949/) (2021)

## Abstract (arXiv)

Discriminative analysis in neuroimaging by means of deep/machine learning techniques is usually tested with validation techniques, whereas the associated statistical significance remains largely under-developed due to their computational complexity. In this work, a non-parametric framework is proposed that estimates the statistical significance of classifications using deep learning architectures. In particular, a combination of autoencoders (AE) and support vector machines (SVM) is applied to: (i) a one-condition, within-group designs often of normal controls (NC) and; (ii) a two-condition, between-group designs which contrast, for example, Alzheimer's disease (AD) patients with NC (the extension to multi-class analyses is also included). A random-effects inference based on a label permutation test is proposed in both studies using cross-validation (CV) and resubstitution with upper bound correction (RUB) as validation methods. This allows both false positives and classifier overfitting to be detected as well as estimating the statistical power of the test. Several experiments were carried out using the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset, the Dominantly Inherited Alzheimer Network (DIAN) dataset, and a MCI prediction dataset. We found in the permutation test that CV and RUB methods offer a false positive rate close to the significance level and an acceptable statistical power (although lower using cross-validation). A large separation between training and test accuracies using CV was observed, especially in one-condition designs. This implies a low generalization ability as the model fitted in training is not informative with respect to the test set. We propose as solution by applying RUB, whereby similar results are obtained to those of the CV test set, but considering the whole set and with a lower computational cost per iteration.