# Multi-Modality in Music: Predicting Emotion in Music from High-Level Audio Features and Lyrics

arXiv: https://arxiv.org/abs/2302.13321 · HTML: https://mlsanity.com/paper/2302.13321/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 1 of its 62 references that are in the map.
- First posted on arXiv 26 Feb 2023.

## Builds on (most cited first, 1 of 1)

- [Music Mood Detection Based On Audio And Lyrics With Deep Neural Net](https://mlsanity.com/paper/1809.07276/) (2018)

## Built on by (most cited first, 2 of 2)

- [Are We There Yet? A Brief Survey of Music Emotion Prediction Datasets, Models and Outstanding Challenges](https://mlsanity.com/paper/2406.08809/) (2024)
- [MusicLIME: Explainable Multimodal Music Understanding](https://mlsanity.com/paper/2409.10496/) (2024)

## Abstract (arXiv)

This paper aims to test whether a multi-modal approach for music emotion recognition (MER) performs better than a uni-modal one on high-level song features and lyrics. We use 11 song features retrieved from the Spotify API, combined lyrics features including sentiment, TF-IDF, and Anew to predict valence and arousal (Russell, 1980) scores on the Deezer Mood Detection Dataset (DMDD) (Delbouys et al., 2018) with 4 different regression models. We find that out of the 11 high-level song features, mainly 5 contribute to the performance, multi-modal features do better than audio alone when predicting valence. We made our code publically available.