# A Hybrid Framework for Song Lyric Annotation Based on Human-LLM Alignment

arXiv: https://arxiv.org/abs/2606.29273 · HTML: https://mlsanity.com/paper/2606.29273/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 38 references that are in the map.
- First posted on arXiv 28 Jun 2026.

## Builds on (most cited first, 8 of 8)

- [GoEmotions: A Dataset of Fine-Grained Emotions](https://mlsanity.com/paper/2005.00547/) (2020)
- [Crowdsourcing a Word-Emotion Association Lexicon](https://mlsanity.com/paper/1308.6297/) (2013)
- [EMOPIA: A Multi-Modal Pop Piano Dataset For Emotion Recognition and Emotion-based Music Generation](https://mlsanity.com/paper/2108.01374/) (2021)
- [Investigating Societal Biases in a Poetry Composition System](https://mlsanity.com/paper/2011.02686/) (2020)
- [Transformer-based approach towards music emotion recognition from lyrics](https://mlsanity.com/paper/2101.02051/) (2021)
- [Tollywood Emotions: Annotation of Valence-Arousal in Telugu Song Lyrics](https://mlsanity.com/paper/2303.09364/) (2023)
- [Emotion4MIDI: a Lyrics-based Emotion-Labeled Symbolic Music Dataset](https://mlsanity.com/paper/2307.14783/) (2023)
- [A Survey on Multimodal Music Emotion Recognition](https://mlsanity.com/paper/2504.18799/) (2025)

## Abstract (arXiv)

Emotion recognition of song lyrics is a challenging task since lyrics may not necessarily align with the overall emotion of a song. As a result, lyrics annotation remains largely underexplored. Drawing inspiration from research in large language model (LLM) assisted annotation, we examine the alignment between humans and LLMs for annotation of lyrics by creating a new sentence-level dataset of lyrics. Our observations highlight the subjectivity of the task and the inherent challenges. Following this, we present a hybrid annotation framework that optimizes human and LLM annotation by predicting potential misalignment in annotation.