# SimLabel: Similarity-Weighted Iterative Framework for Multi-annotator Learning with Missing Annotations

arXiv: https://arxiv.org/abs/2504.09525 · HTML: https://mlsanity.com/paper/2504.09525/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Apr 2025.

## Built on by (most cited first, 1 of 1)

- [A Unified Evaluation Framework for Multi-Annotator Tendency Learning](https://mlsanity.com/paper/2508.10393/) (2025)

## Abstract (arXiv)

Multi-annotator learning (MAL) aims to model annotator-specific labeling patterns. However, existing methods face a critical challenge: they simply skip updating annotator-specific model parameters when encountering missing labels, i.e., a common scenario in real-world crowdsourced datasets where each annotator labels only small subsets of samples. This leads to inefficient data utilization and overfitting risks. To this end, we propose a novel similarity-weighted semi-supervised learning framework (SimLabel) that leverages inter-annotator similarities to generate weighted soft labels for missing annotations, enabling the utilization of unannotated samples rather than skipping them entirely. We further introduce a confidence-based iterative refinement mechanism that combines maximum probability with entropy-based uncertainty to prioritize predicted high-quality pseudo-labels to impute missing labels, jointly enhancing similarity estimation and model performance over time. For evaluation, we contribute a new multimodal multi-annotator dataset, AMER2, with high and more variable missing rates, reflecting real-world annotation sparsity and enabling evaluation across different sparsity levels.