# Towards Unified Music Emotion Recognition across Dimensional and Categorical Models

arXiv: https://arxiv.org/abs/2502.03979 · HTML: https://mlsanity.com/paper/2502.03979/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 56 references that are in the map.
- First posted on arXiv 6 Feb 2025.

## Builds on (most cited first, 16 of 16)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [CNN Architectures for Large-Scale Audio Classification](https://mlsanity.com/paper/1609.09430/) (2016)
- [MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Training](https://mlsanity.com/paper/2306.00107/) (2023)
- [EMOPIA: A Multi-Modal Pop Piano Dataset For Emotion Recognition and Emotion-based Music Generation](https://mlsanity.com/paper/2108.01374/) (2021)
- [Learning to Generate Music With Sentiment](https://mlsanity.com/paper/2103.06125/) (2021)
- [A Bi-directional Transformer for Musical Chord Recognition](https://mlsanity.com/paper/1907.02698/) (2019)
- [Generating Lead Sheets with Affect: A Novel Conditional seq2seq Framework](https://mlsanity.com/paper/2104.13056/) (2021)
- [Are We There Yet? A Brief Survey of Music Emotion Prediction Datasets, Models and Outstanding Challenges](https://mlsanity.com/paper/2406.08809/) (2024)
- [Constructing Time-Series Momentum Portfolios with Deep Multi-Task Learning](https://mlsanity.com/paper/2306.13661/) (2023)
- [Multitask Emotion Recognition Model with Knowledge Distillation and Task Discriminator](https://mlsanity.com/paper/2203.13072/) (2022)
- [CNN based music emotion classification](https://mlsanity.com/paper/1704.05665/) (2017)
- [Semi-supervised music emotion recognition using noisy student training and harmonic pitch class profiles](https://mlsanity.com/paper/2112.00702/) (2021)
- [A Novel Multi-Task Learning Method for Symbolic Music Emotion Recognition](https://mlsanity.com/paper/2201.05782/) (2022)
- [ADFF: Attention Based Deep Feature Fusion Approach for Music Emotion Recognition](https://mlsanity.com/paper/2204.05649/) (2022)
- [Music Tagging with Classifier Group Chains](https://mlsanity.com/paper/2501.05050/) (2025)
- [Leveraging LLM Embeddings for Cross Dataset Label Alignment and Zero Shot Music Emotion Prediction](https://mlsanity.com/paper/2410.11522/) (2024)

## Built on by (most cited first, 7 of 7)

- [MECAT: A Multi-Experts Constructed Benchmark for Fine-Grained Audio Understanding Tasks](https://mlsanity.com/paper/2507.23511/) (2025)
- [Content filtering methods for music recommendation: A review](https://mlsanity.com/paper/2507.02282/) (2025)
- [KARMA-MV: A Benchmark for Causal Question Answering on Music Videos](https://mlsanity.com/paper/2605.08175/) (2026)
- [Constructing Composite Features for Interpretable Music-Tagging](https://mlsanity.com/paper/2603.28644/) (2026)
- [Affective Music Recommendation: A Rollout-Based World Model for Offline Preference Optimization](https://mlsanity.com/paper/2605.28810/) (2026)
- [Mood-Aware Music Recommendation: Integrating User Affective Signals into Ranking Systems](https://mlsanity.com/paper/2606.13858/) (2026)
- [Dissonance Spectrum explicitly models perceptual frequency interactions for better music understanding](https://mlsanity.com/paper/2608.25621/) (2026)

## Abstract (arXiv)

One of the most significant challenges in Music Emotion Recognition (MER) comes from the fact that emotion labels can be heterogeneous across datasets with regard to the emotion representation, including categorical (e.g., happy, sad) versus dimensional labels (e.g., valence-arousal). In this paper, we present a unified multitask learning framework that combines these two types of labels and is thus able to be trained on multiple datasets. This framework uses an effective input representation that combines musical features (i.e., key and chords) and MERT embeddings. Moreover, knowledge distillation is employed to transfer the knowledge of teacher models trained on individual datasets to a student model, enhancing its ability to generalize across multiple tasks. To validate our proposed framework, we conducted extensive experiments on a variety of datasets, including MTG-Jamendo, DEAM, PMEmo, and EmoMusic. According to our experimental results, the inclusion of musical features, multitask learning, and knowledge distillation significantly enhances performance. In particular, our model outperforms the state-of-the-art models, including the best-performing model from the MediaEval 2021 competition on the MTG-Jamendo dataset. Our work makes a significant contribution to MER by allowing the combination of categorical and dimensional emotion labels in one unified framework, thus enabling training across datasets.