# Difficulty-aware Glaucoma Classification with Multi-Rater Consensus Modeling

arXiv: https://arxiv.org/abs/2007.14848 · HTML: https://mlsanity.com/paper/2007.14848/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Jul 2020.

## Built on by (most cited first, 17 of 17)

- [Less Could Be Better: Parameter-efficient Fine-tuning Advances Medical Vision Foundation Models](https://mlsanity.com/paper/2401.12215/) (2024)
- [Label fusion and training methods for reliable representation of inter-rater uncertainty](https://mlsanity.com/paper/2202.07550/) (2022)
- [Learning self-calibrated optic disc and cup segmentation from multi-rater annotations](https://mlsanity.com/paper/2206.05092/) (2022)
- [UNet-2022: Exploring Dynamics in Non-isomorphic Architecture](https://mlsanity.com/paper/2210.15566/) (2022)
- [Quality Assured: Rethinking Annotation Strategies in Imaging AI](https://mlsanity.com/paper/2407.17596/) (2024)
- [Deep Learning and Computer Vision for Glaucoma Detection: A Review](https://mlsanity.com/paper/2307.16528/) (2023)
- [Pathological Image Segmentation with Noisy Labels](https://mlsanity.com/paper/2104.02602/) (2021)
- [Opinions Vary? Diagnosis First!](https://mlsanity.com/paper/2202.06505/) (2022)
- [Calibrate the inter-observer segmentation uncertainty via diagnosis-first principle](https://mlsanity.com/paper/2208.03016/) (2022)
- [Joint Salient Object Detection and Camouflaged Object Detection via Uncertainty-aware Learning](https://mlsanity.com/paper/2307.04651/) (2023)
- [Multi-rater Prompting for Ambiguous Medical Image Segmentation](https://mlsanity.com/paper/2404.07580/) (2024)
- [Bayesian Detector Combination for Object Detection with Crowdsourced Annotations](https://mlsanity.com/paper/2407.07958/) (2024)
- [Label Filling via Mixed Supervision for Medical Image Segmentation from Noisy Annotations](https://mlsanity.com/paper/2410.16057/) (2024)
- [Learning from Subjective Ratings Using Auto-Decoded Deep Latent Embeddings](https://mlsanity.com/paper/2104.05570/) (2021)
- [Example Perplexity](https://mlsanity.com/paper/2203.08813/) (2022)
- [Segmentation-based Information Extraction and Amalgamation in Fundus Images for Glaucoma Detection](https://mlsanity.com/paper/2209.11456/) (2022)
- [Learning from Disagreement: A Group Decision Simulation Framework for Robust Medical Image Segmentation](https://mlsanity.com/paper/2510.10462/) (2025)

## Abstract (arXiv)

Medical images are generally labeled by multiple experts before the final ground-truth labels are determined. Consensus or disagreement among experts regarding individual images reflects the gradeability and difficulty levels of the image. However, when being used for model training, only the final ground-truth label is utilized, while the critical information contained in the raw multi-rater gradings regarding the image being an easy/hard case is discarded. In this paper, we aim to take advantage of the raw multi-rater gradings to improve the deep learning model performance for the glaucoma classification task. Specifically, a multi-branch model structure is proposed to predict the most sensitive, most specifical and a balanced fused result for the input images. In order to encourage the sensitivity branch and specificity branch to generate consistent results for consensus labels and opposite results for disagreement labels, a consensus loss is proposed to constrain the output of the two branches. Meanwhile, the consistency/inconsistency between the prediction results of the two branches implies the image being an easy/hard case, which is further utilized to encourage the balanced fusion branch to concentrate more on the hard cases. Compared with models trained only with the final ground-truth labels, the proposed method using multi-rater consensus information has achieved superior performance, and it is also able to estimate the difficulty levels of individual input images when making the prediction.