# The STOIC2021 COVID-19 AI challenge: applying reusable training methodologies to private data

arXiv: https://arxiv.org/abs/2306.10484 · HTML: https://mlsanity.com/paper/2306.10484/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 98 references that are in the map.
- First posted on arXiv 18 Jun 2023.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [Unified Perceptual Parsing for Scene Understanding](https://mlsanity.com/paper/1807.10221/) (2018)
- [iBOT: Image BERT Pre-Training with Online Tokenizer](https://mlsanity.com/paper/2111.07832/) (2021)
- [The Medical Segmentation Decathlon](https://mlsanity.com/paper/2106.05735/) (2021)
- [CHAOS Challenge -- Combined (CT-MR) Healthy Abdominal Organ Segmentation](https://mlsanity.com/paper/2001.06535/) (2020)
- [REFUGE Challenge: A Unified Framework for Evaluating Automated Methods for Glaucoma Assessment from Fundus Photographs](https://mlsanity.com/paper/1910.03667/) (2019)
- [Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the LUNA16 challenge](https://mlsanity.com/paper/1612.08012/) (2016)
- [The state of the art in kidney and kidney tumor segmentation in contrast-enhanced CT imaging: Results of the KiTS19 Challenge](https://mlsanity.com/paper/1912.01054/) (2019)
- [Models Genesis: Generic Autodidactic Models for 3D Medical Image Analysis](https://mlsanity.com/paper/1908.06912/) (2019)
- [MosMedData: Chest CT Scans With COVID-19 Related Findings Dataset](https://mlsanity.com/paper/2005.06465/) (2020)
- [Mitosis domain generalization in histopathology images -- The MIDOG challenge](https://mlsanity.com/paper/2204.03742/) (2022)
- [Predicting breast tumor proliferation from whole-slide images: the TUPAC16 challenge](https://mlsanity.com/paper/1807.08284/) (2018)
- [Advancing machine learning for MR image reconstruction with an open competition: Overview of the 2019 fastMRI challenge](https://mlsanity.com/paper/2001.02518/) (2020)
- [Common Limitations of Image Processing Metrics: A Picture Story](https://mlsanity.com/paper/2104.05642/) (2021)
- [OpenKBP: The open-access knowledge-based planning grand challenge](https://mlsanity.com/paper/2011.14076/) (2020)
- [Multi-Site Infant Brain Segmentation Algorithms: The iSeg-2019 Challenge](https://mlsanity.com/paper/2007.02096/) (2020)
- [Automated Design of Deep Learning Methods for Biomedical Image Segmentation](https://mlsanity.com/paper/1904.08128/) (2019)
- [ADAM Challenge: Detecting Age-related Macular Degeneration from Fundus Images](https://mlsanity.com/paper/2202.07983/) (2022)
- [MIScnn: A Framework for Medical Image Segmentation with Convolutional Neural Networks and Deep Learning](https://mlsanity.com/paper/1910.09308/) (2019)
- [AGE Challenge: Angle Closure Glaucoma Evaluation in Anterior Segment Optical Coherence Tomography](https://mlsanity.com/paper/2005.02258/) (2020)
- [TADPOLE Challenge: Accurate Alzheimer's disease prediction through crowdsourced forecasting of future data](https://mlsanity.com/paper/2001.09016/) (2020)
- [Relational Modeling for Robust and Efficient Pulmonary Lobe Segmentation in CT Scans](https://mlsanity.com/paper/2004.07443/) (2020)
- [COVID Detection and Severity Prediction with 3D-ConvNeXt and Custom Pretrainings](https://mlsanity.com/paper/2206.15073/) (2022)

## Abstract (arXiv)

Challenges drive the state-of-the-art of automated medical image analysis. The quantity of public training data that they provide can limit the performance of their solutions. Public access to the training methodology for these solutions remains absent. This study implements the Type Three (T3) challenge format, which allows for training solutions on private data and guarantees reusable training methodologies. With T3, challenge organizers train a codebase provided by the participants on sequestered training data. T3 was implemented in the STOIC2021 challenge, with the goal of predicting from a computed tomography (CT) scan whether subjects had a severe COVID-19 infection, defined as intubation or death within one month. STOIC2021 consisted of a Qualification phase, where participants developed challenge solutions using 2000 publicly available CT scans, and a Final phase, where participants submitted their training methodologies with which solutions were trained on CT scans of 9724 subjects. The organizers successfully trained six of the eight Final phase submissions. The submitted codebases for training and running inference were released publicly. The winning solution obtained an area under the receiver operating characteristic curve for discerning between severe and non-severe COVID-19 of 0.815. The Final phase solutions of all finalists improved upon their Qualification phase solutions.HSUXJM-TNZF9CHSUXJM-TNZF9C