# Leveraging LLM Embeddings for Cross Dataset Label Alignment and Zero Shot Music Emotion Prediction

arXiv: https://arxiv.org/abs/2410.11522 · HTML: https://mlsanity.com/paper/2410.11522/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 21 references that are in the map.
- First posted on arXiv 15 Oct 2024.

## Builds on (most cited first, 6 of 6)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Feature Generating Networks for Zero-Shot Learning](https://mlsanity.com/paper/1712.00981/) (2017)
- [MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Training](https://mlsanity.com/paper/2306.00107/) (2023)
- [Are We There Yet? A Brief Survey of Music Emotion Prediction Datasets, Models and Outstanding Challenges](https://mlsanity.com/paper/2406.08809/) (2024)
- [Predicting emotion from music videos: exploring the relative contribution of visual and auditory information to affective responses](https://mlsanity.com/paper/2202.10453/) (2022)

## Built on by (most cited first, 1 of 1)

- [Towards Unified Music Emotion Recognition across Dimensional and Categorical Models](https://mlsanity.com/paper/2502.03979/) (2025)

## Abstract (arXiv)

In this work, we present a novel method for music emotion recognition that leverages Large Language Model (LLM) embeddings for label alignment across multiple datasets and zero-shot prediction on novel categories. First, we compute LLM embeddings for emotion labels and apply non-parametric clustering to group similar labels, across multiple datasets containing disjoint labels. We use these cluster centers to map music features (MERT) to the LLM embedding space. To further enhance the model, we introduce an alignment regularization that enables dissociation of MERT embeddings from different clusters. This further enhances the model's ability to better adaptation to unseen datasets. We demonstrate the effectiveness of our approach by performing zero-shot inference on a new dataset, showcasing its ability to generalize to unseen labels without additional training.