# A new baseline for retinal vessel segmentation: Numerical identification and correction of methodological inconsistencies affecting 100+ papers

arXiv: https://arxiv.org/abs/2111.03853 · HTML: https://mlsanity.com/paper/2111.03853/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 131 references that are in the map.
- First posted on arXiv 6 Nov 2021.

## Builds on (most cited first, 8 of 8)

- [Why rankings of biomedical image analysis competitions should be interpreted with care](https://mlsanity.com/paper/1806.02051/) (2018)
- [DUNet: A deformable network for retinal vessel segmentation](https://mlsanity.com/paper/1811.01206/) (2018)
- [Optimization for Medical Image Segmentation: Theory and Practice when evaluating with Dice Score or Jaccard Index](https://mlsanity.com/paper/2010.13499/) (2020)
- [Deep supervision with additional labels for retinal vessel segmentation task](https://mlsanity.com/paper/1806.02132/) (2018)
- [A Fully Convolutional Neural Network based Structured Prediction Approach Towards the Retinal Vessel Segmentation](https://mlsanity.com/paper/1611.02064/) (2016)
- [A Hierarchical Image Matting Model for Blood Vessel Segmentation in Fundus images](https://mlsanity.com/paper/1701.00892/) (2017)
- [Automated retinal vessel segmentation based on morphological preprocessing and 2D-Gabor wavelets](https://mlsanity.com/paper/1908.04123/) (2019)
- [Automatic Segmentation of Retinal Vasculature](https://mlsanity.com/paper/1707.06323/) (2017)

## Built on by (most cited first, 4 of 4)

- [The RETA Benchmark for Retinal Vascular Tree Analysis](https://mlsanity.com/paper/2111.11658/) (2021)
- [A new dataset for measuring the performance of blood vessel segmentation methods under distribution shifts](https://mlsanity.com/paper/2301.04517/) (2023)
- [Testing the Consistency of Performance Scores Reported for Binary Classification Problems](https://mlsanity.com/paper/2310.12527/) (2023)
- [Enumerating the k-fold configurations in multi-class classification problems](https://mlsanity.com/paper/2401.13843/) (2024)

## Abstract (arXiv)

In the last 15 years, the segmentation of vessels in retinal images has become an intensively researched problem in medical imaging, with hundreds of algorithms published. One of the de facto benchmarking data sets of vessel segmentation techniques is the DRIVE data set. Since DRIVE contains a predefined split of training and test images, the published performance results of the various segmentation techniques should provide a reliable ranking of the algorithms. Including more than 100 papers in the study, we performed a detailed numerical analysis of the coherence of the published performance scores. We found inconsistencies in the reported scores related to the use of the field of view (FoV), which has a significant impact on the performance scores. We attempted to eliminate the biases using numerical techniques to provide a more realistic picture of the state of the art. Based on the results, we have formulated several findings, most notably: despite the well-defined test set of DRIVE, most rankings in published papers are based on non-comparable figures; in contrast to the near-perfect accuracy scores reported in the literature, the highest accuracy score achieved to date is 0.9582 in the FoV region, which is 1% higher than that of human annotators. The methods we have developed for identifying and eliminating the evaluation biases can be easily applied to other domains where similar problems may arise.