mlsanityData to 14 Oct 2026

Bias-Corrected Ceilings of Emotion Predictability from Human Label Variation Based on Instance-Level Fano Bounds

arXiv 2608.15619 ↗First posted 16 Aug 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Emotion recognition from text keeps improving on benchmarks, yet whether an accuracy ceiling has been reached is seldom asked with discipline. Our aim is not to pin this ceiling to a single number, but to quantify how far it depends on finite annotation, estimator choice, annotation noise, and the evaluation protocol, and thereby to discipline how confidently saturation can be claimed. We propose Bias-corrected Affective Ceiling Estimation (BACE), an analysis framework that estimates a bias-corrected ceiling, separates irreducible from reducible error, and disciplines the resulting claims. An anchored Dirichlet-mixture empirical Bayes estimator, bracketed between plug-in and NSB, recovers the human-consensus distribution; an annotator split, a noise deconvolution, and a fixed claim gate then attribute error without circularity. Methodologically, unconstrained point estimates place reachability anywhere from 0.38 to 1.03, so saturation cannot be decided by any single estimator. Substantively, the only assertion passing the claim gate is that at least about 33% of a representative classifier's error on GoEmotions is irreducible, with the same pattern recurring on offensiveness and irony.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 9 of its 31 references that are in the map. First posted on arXiv 16 Aug 2026.

Builds on

9 of 31 references.

References · 9

  1. 2020GoEmotions: A Dataset of Fine-Grained Emotionscited by 486
  2. 2021Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotationscited by 314
  3. 2019Human uncertainty makes classification more robustcited by 235
  4. 2020What Can We Learn from Collective Human Opinions on Natural Language Inference Data?cited by 153
  5. 2021Agreeing to Disagree: Annotating Offensive Language Datasets with Annotators' Disagreementcited by 72
  6. 2022Stop Measuring Calibration When Humans Disagreecited by 71
  7. 2023SemEval-2023 Task 11: Learning With Disagreements (LeWiDi)cited by 52
  8. 2022Is the Performance of My Deep Network Too Good to Be True? A Direct Approach to Estimating the Bayes Error in Binary Classificationcited by 26
  9. 2025Practical estimation of the optimal classification error with soft labels and calibrationcited by 6

Built on by

Not cited yet.