DeMeVa at LeWiDi-2025: Modeling Perspectives with In-Context Learning and Label Distribution Learning
Not on the map. Why?
Abstract (arXiv)
This system paper presents the DeMeVa team's approaches to the third edition of the Learning with Disagreements shared task (LeWiDi 2025; Leonardelli et al., 2025). We explore two directions: in-context learning (ICL) with large language models, where we compare example sampling strategies; and label distribution learning (LDL) methods with RoBERTa (Liu et al., 2019b), where we evaluate several fine-tuning methods. Our contributions are twofold: (1) we show that ICL can effectively predict annotator-specific annotations (perspectivist annotations), and that aggregating these predictions into soft labels yields competitive performance; and (2) we argue that LDL methods are promising for soft label predictions and merit further exploration by the perspectivist community.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 29 references that are in the map. First posted on arXiv 11 Sep 2025.
Builds on
19 of 29 references.
References · 19
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
- 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
- 2022A Survey on In-context Learningcited by 721
- 2020Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillationcited by 593
- 2022The Unreliability of Explanations in Few-shot Prompting for Textual Reasoningcited by 169
- 2024Quantifying the Persona Effect in LLM Simulationscited by 169
- 2020What Can We Learn from Collective Human Opinions on Natural Language Inference Data?cited by 153
- 2021Toward a Perspectivist Turn in Ground Truthing for Predictive Computingcited by 130
- 2014Label Distribution Learningcited by 124
- 2024Benchmarking Distributional Alignment of Large Language Modelscited by 61
- 2024VariErr NLI: Separating Annotation Error from Human Label Variationcited by 42
7 more
- 2023Sociodemographic Prompting is Not Yet an Effective Approach for Simulating Subjective Judgments with LLMscited by 36
- 2025Value Profiles for Encoding Human Variationcited by 29
- 2021Global Explainability of BERT-Based Evaluation Metrics by Disentangling along Linguistic Factorscited by 25
- 2024Transparent and Scrutable Recommendations Using Natural Language User Profilescited by 24
- 2022On Natural Language User Profiles for Transparent and Scrutable Recommendationcited by 23
- 2023Ecologically Valid Explanations for Label Variation in NLIcited by 16
- 2024ARTICLE: Annotator Reliability Through In-Context Learningcited by 6
Built on by
Not cited yet.