# Biomedical image analysis competitions: The state of current participation practice

arXiv: https://arxiv.org/abs/2212.08568 · HTML: https://mlsanity.com/paper/2212.08568/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 47 references that are in the map.
- First posted on arXiv 16 Dec 2022.

## Builds on (most cited first, 3 of 3)

- [BIAS: Transparent reporting of biomedical image analysis challenges](https://mlsanity.com/paper/1910.04071/) (2019)
- [CholecTriplet2021: A benchmark challenge for surgical action triplet recognition](https://mlsanity.com/paper/2204.04746/) (2022)
- [Assessing generalisability of deep learning-based polyp detection and segmentation methods through a computer vision challenge](https://mlsanity.com/paper/2202.12031/) (2022)

## Built on by (most cited first, 20 of 20)

- [Learning Multi-modal Representations by Watching Hundreds of Surgical Video Lectures](https://mlsanity.com/paper/2307.15220/) (2023)
- [Correlation-aware Coarse-to-fine MLPs for Deformable Medical Image Registration](https://mlsanity.com/paper/2406.00123/) (2024)
- [MedShapeNet -- A Large-Scale Dataset of 3D Medical Shapes for Computer Vision](https://mlsanity.com/paper/2308.16139/) (2023)
- [Why is the winner the best?](https://mlsanity.com/paper/2303.17719/) (2023)
- [Modality-Agnostic Structural Image Representation Learning for Deformable Multi-Modality Medical Image Registration](https://mlsanity.com/paper/2402.18933/) (2024)
- [Multi-class point cloud completion networks for 3D cardiac anatomy reconstruction from cine magnetic resonance images](https://mlsanity.com/paper/2307.08535/) (2023)
- [DeFormer: Integrating Transformers with Deformable Models for 3D Shape Abstraction from a Single Image](https://mlsanity.com/paper/2309.12594/) (2023)
- [Surgical Visual Understanding (SurgVU) Dataset](https://mlsanity.com/paper/2501.09209/) (2025)
- [Merging-Diverging Hybrid Transformer Networks for Survival Prediction in Head and Neck Cancer](https://mlsanity.com/paper/2307.03427/) (2023)
- [Jumpstarting Surgical Computer Vision](https://mlsanity.com/paper/2312.05968/) (2023)
- [In the Picture: Medical Imaging Datasets, Artifacts, and their Living Review](https://mlsanity.com/paper/2501.10727/) (2025)
- [AutoFuse: Automatic Fusion Networks for Deformable Medical Image Registration](https://mlsanity.com/paper/2309.05271/) (2023)
- [Full-resolution MLPs Empower Medical Dense Prediction](https://mlsanity.com/paper/2311.16707/) (2023)
- [AdaMSS: Adaptive Multi-Modality Segmentation-to-Survival Learning for Survival Outcome Prediction from PET/CT Images](https://mlsanity.com/paper/2305.09946/) (2023)
- [SegHeD: Segmentation of Heterogeneous Data for Multiple Sclerosis Lesions with Anatomical Constraints](https://mlsanity.com/paper/2410.01766/) (2024)
- [[Citation needed] Data usage and citation practices in medical imaging conferences](https://mlsanity.com/paper/2402.03003/) (2024)
- [Foundation Model for Whole-Heart Segmentation: Leveraging Student-Teacher Learning in Multi-Modal Medical Imaging](https://mlsanity.com/paper/2503.19005/) (2025)
- [Instance Segmentation XXL-CT Challenge of a Historic Airplane](https://mlsanity.com/paper/2402.02928/) (2024)
- [Toward Next-generation Medical Vision Backbones: Modeling Finer-grained Long-range Visual Dependency](https://mlsanity.com/paper/2509.11328/) (2025)
- [Similarity Weighted Aggregation with Global Differential Privacy for Federated Brain Lesion Segmentation](https://mlsanity.com/paper/2608.00872/) (2026)

## Abstract (arXiv)

The number of international benchmarking competitions is steadily increasing in various fields of machine learning (ML) research and practice. So far, however, little is known about the common practice as well as bottlenecks faced by the community in tackling the research questions posed. To shed light on the status quo of algorithm development in the specific field of biomedical imaging analysis, we designed an international survey that was issued to all participants of challenges conducted in conjunction with the IEEE ISBI 2021 and MICCAI 2021 conferences (80 competitions in total). The survey covered participants' expertise and working environments, their chosen strategies, as well as algorithm characteristics. A median of 72% challenge participants took part in the survey. According to our results, knowledge exchange was the primary incentive (70%) for participation, while the reception of prize money played only a minor role (16%). While a median of 80 working hours was spent on method development, a large portion of participants stated that they did not have enough time for method development (32%). 25% perceived the infrastructure to be a bottleneck. Overall, 94% of all solutions were deep learning-based. Of these, 84% were based on standard architectures. 43% of the respondents reported that the data samples (e.g., images) were too large to be processed at once. This was most commonly addressed by patch-based training (69%), downsampling (37%), and solving 3D analysis tasks as a series of 2D tasks. K-fold cross-validation on the training set was performed by only 37% of the participants and only 50% of the participants performed ensembling based on multiple identical models (61%) or heterogeneous models (39%). 48% of the respondents applied postprocessing steps.