# Ten years of image analysis and machine learning competitions in dementia

arXiv: https://arxiv.org/abs/2112.07922 · HTML: https://mlsanity.com/paper/2112.07922/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 48 references that are in the map.
- First posted on arXiv 15 Dec 2021.

## Builds on (most cited first, 8 of 8)

- [Accounting for Variance in Machine Learning Benchmarks](https://mlsanity.com/paper/2103.03098/) (2021)
- [Convolutional Neural Networks for Classification of Alzheimer's Disease: Overview and Reproducible Evaluation](https://mlsanity.com/paper/1904.07773/) (2019)
- [Alzheimer's Dementia Recognition through Spontaneous Speech: The ADReSS Challenge](https://mlsanity.com/paper/2004.06833/) (2020)
- [Why rankings of biomedical image analysis competitions should be interpreted with care](https://mlsanity.com/paper/1806.02051/) (2018)
- [Predicting brain age with deep learning from raw imaging data results in a reliable and heritable biomarker](https://mlsanity.com/paper/1612.02572/) (2016)
- [Improving 3D convolutional neural network comprehensibility via interactive visualization of relevance maps: Evaluation in Alzheimer's disease](https://mlsanity.com/paper/2012.10294/) (2020)
- [How can we learn (more) from challenges? A statistical approach to driving future algorithm development](https://mlsanity.com/paper/2106.09302/) (2021)
- [A Framework for Challenge Design: Insight and Deployment Challenges to Address Medical Image Analysis Problems](https://mlsanity.com/paper/1911.08531/) (2019)

## Built on by (most cited first, 5 of 5)

- [Data-Driven Disease Progression Modelling](https://mlsanity.com/paper/2211.05786/) (2022)
- [Parameter inference from a non-stationary unknown process](https://mlsanity.com/paper/2407.08987/) (2024)
- [Computer-aided diagnosis and prediction in brain disorders](https://mlsanity.com/paper/2206.14683/) (2022)
- [Artificial Intelligence for Dementia Research Methods Optimization](https://mlsanity.com/paper/2303.01949/) (2023)
- [The Explanation Necessity for Healthcare AI](https://mlsanity.com/paper/2406.00216/) (2024)

## Abstract (arXiv)

Machine learning methods exploiting multi-parametric biomarkers, especially based on neuroimaging, have huge potential to improve early diagnosis of dementia and to predict which individuals are at-risk of developing dementia. To benchmark algorithms in the field of machine learning and neuroimaging in dementia and assess their potential for use in clinical practice and clinical trials, seven grand challenges have been organized in the last decade. The seven grand challenges addressed questions related to screening, clinical status estimation, prediction and monitoring in (pre-clinical) dementia. There was little overlap in clinical questions, tasks and performance metrics. Whereas this aids providing insight on a broad range of questions, it also limits the validation of results across challenges. The validation process itself was mostly comparable between challenges, using similar methods for ensuring objective comparison, uncertainty estimation and statistical testing. In general, winning algorithms performed rigorous data preprocessing and combined a wide range of input features. Despite high state-of-the-art performances, most of the methods evaluated by the challenges are not clinically used. To increase impact, future challenges could pay more attention to statistical analysis of which factors relate to higher performance, to clinical questions beyond Alzheimer's disease, and to using testing data beyond the Alzheimer's Disease Neuroimaging Initiative. Grand challenges would be an ideal venue for assessing the generalizability of algorithm performance to unseen data of other cohorts. Key for increasing impact in this way are larger testing data sizes, which could be reached by sharing algorithms rather than data to exploit data that cannot be shared.