# What You Feel Is Not What They See: On Predicting Self-Reported Emotion from Third-Party Observer Labels

arXiv: https://arxiv.org/abs/2601.21130 · HTML: https://mlsanity.com/paper/2601.21130/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 23 references that are in the map.
- First posted on arXiv 29 Jan 2026.

## Builds on (most cited first, 9 of 9)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [gpt-oss-120b & gpt-oss-20b Model Card](https://mlsanity.com/paper/2508.10925/) (2025)
- [Dawn of the transformer era in speech emotion recognition: closing the valence gap](https://mlsanity.com/paper/2203.07378/) (2022)
- [MuSE-ing on the Impact of Utterance Ordering On Crowdsourced Emotion Annotations](https://mlsanity.com/paper/1903.11672/) (2019)
- [BERT-ERC: Fine-tuning BERT is Enough for Emotion Recognition in Conversation](https://mlsanity.com/paper/2301.06745/) (2023)
- [Can Third-parties Read Our Emotions?](https://mlsanity.com/paper/2504.18673/) (2025)
- [Efficient Finetuning for Dimensional Speech Emotion Recognition in the Age of Transformers](https://mlsanity.com/paper/2503.03756/) (2025)

## Built on by (most cited first, 2 of 2)

- [Precision-Varying Prediction (PVP): Robustifying ASR systems against adversarial attacks](https://mlsanity.com/paper/2603.22590/) (2026)
- [A Cross Community Agenda for Speech AI](https://mlsanity.com/paper/2609.13168/) (2026)

## Abstract (arXiv)

Self-reported emotion labels capture internal experience, while third-party labels reflect external perception. These perspectives often diverge, limiting the applicability of third-party-trained models to self-report contexts. This gap is critical in mental health, where accurate self-report modeling is essential for guiding intervention. We present the first cross-corpus evaluation of third-party-trained models on self-reports. We find activation unpredictable (CCC approximately 0) and valence moderately predictable (CCC approximately 0.3). Crucially, when content is personally significant to the speaker, models achieve high performance for valence (CCC approximately 0.6-0.8). Our findings point to personal significance as a key pathway for aligning external perception with internal experience and underscore the challenge of self-report activation modeling.