# The NeurIPS 2023 Machine Learning for Audio Workshop: Affective Audio Benchmarks and Novel Data

arXiv: https://arxiv.org/abs/2403.14048 · HTML: https://mlsanity.com/paper/2403.14048/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 35 references that are in the map.
- First posted on arXiv 21 Mar 2024.

## Builds on (most cited first, 16 of 16)

- [Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers](https://mlsanity.com/paper/2301.02111/) (2023)
- [MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations](https://mlsanity.com/paper/1810.02508/) (2018)
- [End-to-End Multimodal Emotion Recognition using Deep Neural Networks](https://mlsanity.com/paper/1704.08619/) (2017)
- [The ICML 2022 Expressive Vocalizations Workshop and Competition: Recognizing, Generating, and Personalizing Vocal Bursts](https://mlsanity.com/paper/2205.01780/) (2022)
- [The ACM Multimedia 2023 Computational Paralinguistics Challenge: Emotion Share & Requests](https://mlsanity.com/paper/2304.14882/) (2023)
- [The ACII 2022 Affective Vocal Bursts Workshop & Competition: Understanding a critically understudied modality of emotional expression](https://mlsanity.com/paper/2207.03572/) (2022)
- [Msanii: High Fidelity Music Synthesis on a Shoestring Budget](https://mlsanity.com/paper/2301.06468/) (2023)
- [Redundancy Reduction Twins Network: A Training framework for Multi-output Emotion Regression](https://mlsanity.com/paper/2206.09142/) (2022)
- [Exploring the Effectiveness of Self-supervised Learning and Classifier Chains in Emotion Recognition of Nonverbal Vocalizations](https://mlsanity.com/paper/2206.10695/) (2022)
- [Synthesizing Personalized Non-speech Vocalization from Discrete Speech Representations](https://mlsanity.com/paper/2206.12662/) (2022)
- [Jointly Predicting Emotion, Age, and Country Using Pre-Trained Acoustic Embedding](https://mlsanity.com/paper/2207.10333/) (2022)
- [Self-Supervised Attention Networks and Uncertainty Loss Weighting for Multi-Task Emotion Recognition on Vocal Bursts](https://mlsanity.com/paper/2209.07384/) (2022)
- [Self-Relation Attention and Temporal Awareness for Emotion Recognition via Vocal Burst](https://mlsanity.com/paper/2209.07629/) (2022)
- [Classification of Vocal Bursts for ACII 2022 A-VB-Type Competition using Convolutional Neural Networks and Deep Acoustic Embeddings](https://mlsanity.com/paper/2209.14842/) (2022)
- [Fine-tuning Wav2vec for Vocal-burst Emotion Recognition](https://mlsanity.com/paper/2210.00263/) (2022)
- [A Hierarchical Regression Chain Framework for Affective Vocal Burst Recognition](https://mlsanity.com/paper/2303.08027/) (2023)

## Abstract (arXiv)

The NeurIPS 2023 Machine Learning for Audio Workshop brings together machine learning (ML) experts from various audio domains. There are several valuable audio-driven ML tasks, from speech emotion recognition to audio event detection, but the community is sparse compared to other ML areas, e.g., computer vision or natural language processing. A major limitation with audio is the available data; with audio being a time-dependent modality, high-quality data collection is time-consuming and costly, making it challenging for academic groups to apply their often state-of-the-art strategies to a larger, more generalizable dataset. In this short white paper, to encourage researchers with limited access to large-datasets, the organizers first outline several open-source datasets that are available to the community, and for the duration of the workshop are making several propriety datasets available. Namely, three vocal datasets, Hume-Prosody, Hume-VocalBurst, an acted emotional speech dataset Modulate-Sonata, and an in-game streamer dataset Modulate-Stream. We outline the current baselines on these datasets but encourage researchers from across audio to utilize them outside of the initial baseline tasks.