# Creating an African American-Sounding TTS: Guidelines, Technical Challenges,and Surprising Evaluations

arXiv: https://arxiv.org/abs/2403.11209 · HTML: https://mlsanity.com/paper/2403.11209/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 62 references that are in the map.
- First posted on arXiv 17 Mar 2024.

## Builds on (most cited first, 8 of 8)

- [Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions](https://mlsanity.com/paper/1712.05884/) (2017)
- [LPCNet: Improving Neural Speech Synthesis Through Linear Prediction](https://mlsanity.com/paper/1810.11846/) (2018)
- [A Survey of Race, Racism, and Anti-Racism in NLP](https://mlsanity.com/paper/2106.11410/) (2021)
- [Non-Attentive Tacotron: Robust and Controllable Neural TTS Synthesis Including Unsupervised Duration Modeling](https://mlsanity.com/paper/2010.04301/) (2020)
- [Conversational End-to-End TTS for Voice Agent](https://mlsanity.com/paper/2005.10438/) (2020)
- [Quality Control in Crowdsourcing: A Survey of Quality Attributes, Assessment Techniques and Assurance Actions](https://mlsanity.com/paper/1801.02546/) (2018)
- [Robots Racialized in the Likeness of Marginalized Social Identities are Subject to Greater Dehumanization than those racialized as White](https://mlsanity.com/paper/1808.00320/) (2018)
- [A Neural TTS System with Parallel Prosody Transfer from Unseen Speakers](https://mlsanity.com/paper/2309.11487/) (2023)

## Built on by (most cited first, 4 of 4)

- [The Voice: Lessons on Trustworthy Conversational Agents from "Dune"](https://mlsanity.com/paper/2407.18928/) (2024)
- [Finding A Voice: Exploring the Potential of African American Dialect and Voice Generation for Chatbots](https://mlsanity.com/paper/2501.03441/) (2025)
- [My LLM might Mimic AAE -- But When Should it?](https://mlsanity.com/paper/2502.04564/) (2025)
- [Sexualised synthetic personas encode and amplify gendered power asymmetries through voice](https://mlsanity.com/paper/2606.21366/) (2026)

## Abstract (arXiv)

Representations of AI agents in user interfaces and robotics are predominantly White, not only in terms of facial and skin features, but also in the synthetic voices they use. In this paper we explore some unexpected challenges in the representation of race we found in the process of developing an U.S. English Text-to-Speech (TTS) system aimed to sound like an educated, professional, regional accent-free African American woman. The paper starts by presenting the results of focus groups with African American IT professionals where guidelines and challenges for the creation of a representative and appropriate TTS system were discussed and gathered, followed by a discussion about some of the technical difficulties faced by the TTS system developers. We then describe two studies with U.S. English speakers where the participants were not able to attribute the correct race to the African American TTS voice while overwhelmingly correctly recognizing the race of a White TTS system of similar quality. A focus group with African American IT workers not only confirmed the representativeness of the African American voice we built, but also suggested that the surprising recognition results may have been caused by the inability or the latent prejudice from non-African Americans to associate educated, non-vernacular, professionally-sounding voices to African American people.