# Lyric document embeddings for music tagging

arXiv: https://arxiv.org/abs/2112.11436 · HTML: https://mlsanity.com/paper/2112.11436/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 33 references that are in the map.
- First posted on arXiv 29 Nov 2021.

## Builds on (most cited first, 8 of 8)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Distributed Representations of Sentences and Documents](https://mlsanity.com/paper/1405.4053/) (2014)
- [Deep Learning for Sentiment Analysis : A Survey](https://mlsanity.com/paper/1801.07883/) (2018)
- [Music Mood Detection Based On Audio And Lyrics With Deep Neural Net](https://mlsanity.com/paper/1809.07276/) (2018)
- [Lyrics-Based Music Genre Classification Using a Hierarchical Attention Network](https://mlsanity.com/paper/1707.04678/) (2017)

## Built on by (most cited first, 2 of 2)

- [Lyrics Matter: Exploiting the Power of Learnt Representations for Music Popularity Prediction](https://mlsanity.com/paper/2512.05508/) (2025)
- [IMMENSE: Inductive Multi-perspective User Classification in Social Networks](https://mlsanity.com/paper/2608.05259/) (2026)

## Abstract (arXiv)

We present an empirical study on embedding the lyrics of a song into a fixed-dimensional feature for the purpose of music tagging. Five methods of computing token-level and four methods of computing document-level representations are trained on an industrial-scale dataset of tens of millions of songs. We compare simple averaging of pretrained embeddings to modern recurrent and attention-based neural architectures. Evaluating on a wide range of tagging tasks such as genre classification, explicit content identification and era detection, we find that averaging word embeddings outperform more complex architectures in many downstream metrics.