# The Voice: Lessons on Trustworthy Conversational Agents from "Dune"

arXiv: https://arxiv.org/abs/2407.18928 · HTML: https://mlsanity.com/paper/2407.18928/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 27 references that are in the map.
- First posted on arXiv 10 Jul 2024.

## Builds on (most cited first, 7 of 7)

- [An Overview of Catastrophic AI Risks](https://mlsanity.com/paper/2306.12001/) (2023)
- [Does GPT-4 pass the Turing test?](https://mlsanity.com/paper/2310.20216/) (2023)
- [Computers as Bad Social Actors: Dark Patterns and Anti-Patterns in Interfaces that Act Socially](https://mlsanity.com/paper/2302.04720/) (2023)
- [I am Definitely Manipulated, Even When I am Aware of it. It s Ridiculous! -- Dark Patterns from the End-User Perspective](https://mlsanity.com/paper/2104.12653/) (2021)
- [Impact of Voice Fidelity on Decision Making: A Potential Dark Pattern?](https://mlsanity.com/paper/2402.07010/) (2024)
- [Creating an African American-Sounding TTS: Guidelines, Technical Challenges,and Surprising Evaluations](https://mlsanity.com/paper/2403.11209/) (2024)
- [Killer Apps: Low-Speed, Large-Scale AI Weapons](https://mlsanity.com/paper/2402.01663/) (2024)

## Built on by (most cited first, 1 of 1)

- [Toward Metaphor-Fluid Conversation Design for Voice User Interfaces](https://mlsanity.com/paper/2502.11554/) (2025)

## Abstract (arXiv)

The potential for untrustworthy conversational agents presents a significant threat for covert social manipulation. Taking inspiration from Frank Herbert's "Dune", where the Bene Gesserit Sisterhood uses the Voice for influence, manipulation, and control of people, we explore how generative AI provides a way to implement individualized influence at industrial scales. Already, these models can manipulate communication across text, image, speech, and most recently video. They are rapidly becoming affordable enough for any organization of even moderate means to train and deploy. If employed by malicious actors, they risk becoming powerful tools for shaping public opinion, sowing discord, and undermining organizations from companies to governments. As researchers and developers, it is crucial to recognize the potential for such weaponization and to explore strategies for prevention, detection, and defense against these emerging forms of sociotechnical manipulation.