# Harmonized Feature Conditioning and Frequency-Prompt Personalization for Multi-Rater Medical Segmentation

arXiv: https://arxiv.org/abs/2605.08210 · HTML: https://mlsanity.com/paper/2605.08210/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 37 of its 45 references that are in the map.
- First posted on arXiv 6 May 2026.

## Builds on (most cited first, 30 of 37)

- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels](https://mlsanity.com/paper/1804.06872/) (2018)
- [Kvasir-SEG: A Segmented Polyp Dataset](https://mlsanity.com/paper/1911.07069/) (2019)
- [Symmetric Cross Entropy for Robust Learning with Noisy Labels](https://mlsanity.com/paper/1908.06112/) (2019)
- [Combating noisy labels by agreement: A joint training method with co-regularization](https://mlsanity.com/paper/2003.02752/) (2020)
- [A Probabilistic U-Net for Segmentation of Ambiguous Images](https://mlsanity.com/paper/1806.05034/) (2018)
- [Visual Attention Network](https://mlsanity.com/paper/2202.09741/) (2022)
- [Not-so-supervised: a survey of semi-supervised, multi-instance, and transfer learning in medical image analysis](https://mlsanity.com/paper/1804.06353/) (2018)
- [PHiSeg: Capturing Uncertainty in Medical Image Segmentation](https://mlsanity.com/paper/1906.04045/) (2019)
- [Learning From Noisy Labels By Regularized Estimation Of Annotator Confusion](https://mlsanity.com/paper/1902.03680/) (2019)
- [The Cramer Distance as a Solution to Biased Wasserstein Gradients](https://mlsanity.com/paper/1705.10743/) (2017)
- [Stochastic Segmentation Networks: Modelling Spatially Correlated Aleatoric Uncertainty](https://mlsanity.com/paper/2006.06015/) (2020)
- [Ambiguous Medical Image Segmentation using Diffusion Models](https://mlsanity.com/paper/2304.04745/) (2023)
- [Who Said What: Modeling Individual Labelers Improves Classification](https://mlsanity.com/paper/1703.08774/) (2017)
- [A Hierarchical Probabilistic U-Net for Modeling Multi-Scale Ambiguities](https://mlsanity.com/paper/1905.13077/) (2019)
- [Adaptive Early-Learning Correction for Segmentation from Noisy Annotations](https://mlsanity.com/paper/2110.03740/) (2021)
- [Disentangling Human Error from the Ground Truth in Segmentation of Medical Images](https://mlsanity.com/paper/2007.15963/) (2020)
- [Is segmentation uncertainty useful?](https://mlsanity.com/paper/2103.16265/) (2021)
- [Superpixel-guided Iterative Learning from Noisy Labels for Medical Image Segmentation](https://mlsanity.com/paper/2107.10100/) (2021)
- [Using Soft Labels to Model Uncertainty in Medical Image Segmentation](https://mlsanity.com/paper/2109.12622/) (2021)
- [Probabilistic Modeling of Inter- and Intra-observer Variability in Medical Image Segmentation](https://mlsanity.com/paper/2307.11397/) (2023)
- [D-LEMA: Deep Learning Ensembles from Multiple Annotations -- Application to Skin Lesion Segmentation](https://mlsanity.com/paper/2012.07206/) (2020)
- [That Label's Got Style: Handling Label Style Bias for Uncertain Image Segmentation](https://mlsanity.com/paper/2303.15850/) (2023)
- [QUBIQ: Uncertainty Quantification for Biomedical Image Segmentation Challenge](https://mlsanity.com/paper/2405.18435/) (2024)
- [Laplacian-Former: Overcoming the Limitations of Vision Transformers in Local Texture Detection](https://mlsanity.com/paper/2309.00108/) (2023)
- [Diversified and Personalized Multi-rater Medical Image Segmentation](https://mlsanity.com/paper/2403.13417/) (2024)
- [Ultrasound Nodule Segmentation Using Asymmetric Learning with Simple Clinical Annotation](https://mlsanity.com/paper/2404.14852/) (2024)
- [Transformer-based Annotation Bias-aware Medical Image Segmentation](https://mlsanity.com/paper/2306.01340/) (2023)
- [Learning Robust Medical Image Segmentation from Multi-source Annotations](https://mlsanity.com/paper/2304.00466/) (2023)
- [Confidence Contours: Uncertainty-Aware Annotation for Medical Semantic Segmentation](https://mlsanity.com/paper/2308.07528/) (2023)

## Abstract (arXiv)

Multi-rater medical image segmentation captures the inherent ambiguity of clinical interpretation, where diagnostic boundaries vary across experts and imaging devices. Existing approaches often reduce this diversity to consensus labels or treat rater differences as noise, resulting in overconfident and poorly calibrated models. We propose a harmonized probabilistic framework that disentangles acquisition artifacts from genuine annotator variability through adaptive feature conditioning and frequency-domain personalization. A lightweight Harmonizer Network implicitly models scanner-specific artifacts and performs dynamic feature modulation to standardize latent representations, ensuring that uncertainty reflects anatomy rather than noise. To represent rater-specific styles, we introduce a novel High-Frequency Prompt Modules that operate in the spectral domain to encode annotator-dependent boundary precision and textural sensitivity. These prompts adaptively modulate harmonized features to produce personalized yet anatomically consistent segmentations. Furthermore, a Generalized Energy Distance based regularization aligns the generative distribution with empirical annotation variability, promoting diversity where experts disagree and consensus where they converge. Experiments on LIDC-IDRI and NPC-170 show SOTA aggregated and individualized segmentation, with notable GED reductions and improved Dice scores, especially on noisy cases. Beyond accuracy, the model exhibits clinically meaningful uncertainty. Confidence rises in agreement regions and declines in ambiguous areas, supporting its use as a reliable and interpretable tool for multi-expert clinical workflows.