# Bridging the prosody GAP: Genetic Algorithm with People to efficiently sample emotional prosody

arXiv: https://arxiv.org/abs/2205.04820 · HTML: https://mlsanity.com/paper/2205.04820/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 21 references that are in the map.
- First posted on arXiv 10 May 2022.

## Builds on (most cited first, 6 of 6)

- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [Flowtron: an Autoregressive Flow-based Generative Network for Text-to-Speech Synthesis](https://mlsanity.com/paper/2005.05957/) (2020)
- [Modern Evolution Strategies for Creativity: Fitting Concrete Images and Abstract Concepts](https://mlsanity.com/paper/2109.08857/) (2021)
- [Gibbs Sampling with People](https://mlsanity.com/paper/2008.02595/) (2020)
- [StyleCLIPDraw: Coupling Content and Style in Text-to-Drawing Synthesis](https://mlsanity.com/paper/2111.03133/) (2021)
- [Cross-cultural Mood Perception in Pop Songs and its Alignment with Mood Detection Algorithms](https://mlsanity.com/paper/2108.00768/) (2021)

## Built on by (most cited first, 2 of 2)

- [Characterizing Similarities and Divergences in Conversational Tones in Humans and LLMs by Sampling with People](https://mlsanity.com/paper/2406.04278/) (2024)
- [Analyzing Diffusion as Serial Reproduction](https://mlsanity.com/paper/2209.14821/) (2022)

## Abstract (arXiv)

The human voice effectively communicates a range of emotions with nuanced variations in acoustics. Existing emotional speech corpora are limited in that they are either (a) highly curated to induce specific emotions with predefined categories that may not capture the full extent of emotional experiences, or (b) entangled in their semantic and prosodic cues, limiting the ability to study these cues separately. To overcome this challenge, we propose a new approach called 'Genetic Algorithm with People' (GAP), which integrates human decision and production into a genetic algorithm. In our design, we allow creators and raters to jointly optimize the emotional prosody over generations. We demonstrate that GAP can efficiently sample from the emotional speech space and capture a broad range of emotions, and show comparable results to state-of-the-art emotional speech corpora. GAP is language-independent and supports large crowd-sourcing, thus can support future large-scale cross-cultural research.