# Describing emotions with acoustic property prompts for speech emotion recognition

arXiv: https://arxiv.org/abs/2211.07737 · HTML: https://mlsanity.com/paper/2211.07737/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 21 references that are in the map.
- First posted on arXiv 14 Nov 2022.

## Builds on (most cited first, 6 of 6)

- [MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations](https://mlsanity.com/paper/1810.02508/) (2018)
- [PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition](https://mlsanity.com/paper/1912.10211/) (2019)
- [Clotho: An Audio Captioning Dataset](https://mlsanity.com/paper/1910.09387/) (2019)
- [FSD50K: An Open Dataset of Human-Labeled Sound Events](https://mlsanity.com/paper/2010.00475/) (2020)
- [Audio Retrieval with WavText5K and CLAP Training](https://mlsanity.com/paper/2209.14275/) (2022)
- [What is the ground truth? Reliability of multi-annotator data for audio tagging](https://mlsanity.com/paper/2104.04214/) (2021)

## Built on by (most cited first, 5 of 5)

- [StyleCap: Automatic Speaking-Style Captioning from Speech Based on Speech and Language Self-supervised Learning Models](https://mlsanity.com/paper/2311.16509/) (2023)
- [Interpretable multimodal sentiment analysis based on textual modality descriptions by using large-scale language models](https://mlsanity.com/paper/2305.06162/) (2023)
- [EMO-TTA: Improving Test-Time Adaptation of Audio-Language Models for Speech Emotion Recognition](https://mlsanity.com/paper/2509.25495/) (2025)
- [Solution for Emotion Prediction Competition of Workshop on Emotionally and Culturally Intelligent AI](https://mlsanity.com/paper/2403.17683/) (2024)
- [Toward Efficient Speech Emotion Recognition via Spectral Learning and Attention](https://mlsanity.com/paper/2507.03251/) (2025)

## Abstract (arXiv)

Emotions lie on a broad continuum and treating emotions as a discrete number of classes limits the ability of a model to capture the nuances in the continuum. The challenge is how to describe the nuances of emotions and how to enable a model to learn the descriptions. In this work, we devise a method to automatically create a description (or prompt) for a given audio by computing acoustic properties, such as pitch, loudness, speech rate, and articulation rate. We pair a prompt with its corresponding audio using 5 different emotion datasets. We trained a neural network model using these audio-text pairs. Then, we evaluate the model using one more dataset. We investigate how the model can learn to associate the audio with the descriptions, resulting in performance improvement of Speech Emotion Recognition and Speech Audio Retrieval. We expect our findings to motivate research describing the broad continuum of emotion