# Why is the winner the best? arXiv: https://arxiv.org/abs/2303.17719 ยท HTML: https://mlsanity.com/paper/2303.17719/ ## Key facts - Cited by 21 papers in the map (data through 14 Oct 2026). - Builds on 7 of its 78 references that are in the map. - First posted on arXiv 30 Mar 2023. ## Builds on (most cited first, 7 of 7) - [Why rankings of biomedical image analysis competitions should be interpreted with care](https://mlsanity.com/paper/1806.02051/) (2018) - [Methods and open-source toolkit for analyzing and visualizing challenge results](https://mlsanity.com/paper/1910.05121/) (2019) - [Common Limitations of Image Processing Metrics: A Picture Story](https://mlsanity.com/paper/2104.05642/) (2021) - [BIAS: Transparent reporting of biomedical image analysis challenges](https://mlsanity.com/paper/1910.04071/) (2019) - [Assessing generalisability of deep learning-based polyp detection and segmentation methods through a computer vision challenge](https://mlsanity.com/paper/2202.12031/) (2022) - [Are we using appropriate segmentation metrics? Identifying correlates of human expert perception for CNN training beyond rolling the DICE coefficient](https://mlsanity.com/paper/2103.06205/) (2021) - [Biomedical image analysis competitions: The state of current participation practice](https://mlsanity.com/paper/2212.08568/) (2022) ## Built on by (most cited first, 21 of 21) - [AbdomenAtlas-8K: Annotating 8,000 CT Volumes for Multi-Organ Segmentation in Three Weeks](https://mlsanity.com/paper/2305.09666/) (2023) - [MedShapeNet -- A Large-Scale Dataset of 3D Medical Shapes for Computer Vision](https://mlsanity.com/paper/2308.16139/) (2023) - [Domain Generalization in Computational Pathology: Survey and Guidelines](https://mlsanity.com/paper/2310.19656/) (2023) - [Deep Interactive Segmentation of Medical Images: A Systematic Review and Taxonomy](https://mlsanity.com/paper/2311.13964/) (2023) - [Domain generalization across tumor types, laboratories, and species -- insights from the 2022 edition of the Mitosis Domain Generalization Challenge](https://mlsanity.com/paper/2309.15589/) (2023) - [How Well Do Supervised 3D Models Transfer to Medical Imaging Tasks?](https://mlsanity.com/paper/2501.11253/) (2025) - [nnMobileNet: Rethinking CNN for Retinopathy Research](https://mlsanity.com/paper/2306.01289/) (2023) - [Surgical Visual Understanding (SurgVU) Dataset](https://mlsanity.com/paper/2501.09209/) (2025) - [Multimodal Data Integration for Precision Oncology: Challenges and Future Directions](https://mlsanity.com/paper/2406.19611/) (2024) - [A Review of Bayesian Uncertainty Quantification in Deep Probabilistic Image Segmentation](https://mlsanity.com/paper/2411.16370/) (2024) - [Advances in Automated Fetal Brain MRI Segmentation and Biometry: Insights from the FeTA 2024 Challenge](https://mlsanity.com/paper/2505.02784/) (2025) - [MedAugment: Universal Automatic Data Augmentation Plug-in for Medical Image Analysis](https://mlsanity.com/paper/2306.17466/) (2023) - [Leveraging Imperfection with MEDLEY A Multi-Model Approach Harnessing Bias in Medical AI](https://mlsanity.com/paper/2508.21648/) (2025) - [Segmenting Medical Images with Limited Data](https://mlsanity.com/paper/2407.09189/) (2024) - [ReX-MLE: The Autonomous Agent Benchmark for Medical Imaging Challenges](https://mlsanity.com/paper/2512.17838/) (2025) - [AutoPET Challenge 2023: Sliding Window-based Optimization of U-Net](https://mlsanity.com/paper/2309.12114/) (2023) - [Beyond Knowledge Silos: Task Fingerprinting for Democratization of Medical Imaging AI](https://mlsanity.com/paper/2412.08763/) (2024) - [Auditing Significance, Metric Choice, and Demographic Fairness in Medical AI Challenges](https://mlsanity.com/paper/2512.19091/) (2025) - [Improving Task Instructions for Data Annotators: How Clear Rules and Higher Pay Increase Performance in Data Annotation in the AI Economy](https://mlsanity.com/paper/2312.14565/) (2023) - [SMRVIS: Point cloud extraction from 3-D ultrasound for non-destructive testing](https://mlsanity.com/paper/2306.04668/) (2023) - [Towards Global AI-Driven Cervical Cancer Screening](https://mlsanity.com/paper/2606.15019/) (2026) ## Abstract (arXiv) International benchmarking competitions have become fundamental for the comparative performance assessment of image analysis methods. However, little attention has been given to investigating what can be learnt from these competitions. Do they really generate scientific progress? What are common and successful participation strategies? What makes a solution superior to a competing method? To address this gap in the literature, we performed a multi-center study with all 80 competitions that were conducted in the scope of IEEE ISBI 2021 and MICCAI 2021. Statistical analyses performed based on comprehensive descriptions of the submitted algorithms linked to their rank as well as the underlying participation strategies revealed common characteristics of winning solutions. These typically include the use of multi-task learning (63%) and/or multi-stage pipelines (61%), and a focus on augmentation (100%), image preprocessing (97%), data curation (79%), and postprocessing (66%). The "typical" lead of a winning team is a computer scientist with a doctoral degree, five years of experience in biomedical image analysis, and four years of experience in deep learning. Two core general development strategies stood out for highly-ranked teams: the reflection of the metrics in the method design and the focus on analyzing and handling failure cases. According to the organizers, 43% of the winning algorithms exceeded the state of the art but only 11% completely solved the respective domain problem. The insights of our study could help researchers (1) improve algorithm development strategies when approaching new problems, and (2) focus on open research questions revealed by this work.