# Characterizing Similarities and Divergences in Conversational Tones in Humans and LLMs by Sampling with People

arXiv: https://arxiv.org/abs/2406.04278 · HTML: https://mlsanity.com/paper/2406.04278/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 104 references that are in the map.
- First posted on arXiv 6 Jun 2024.

## Builds on (most cited first, 21 of 21)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Word Translation Without Parallel Data](https://mlsanity.com/paper/1710.04087/) (2017)
- [Challenges in Representation Learning: A report on three machine learning contests](https://mlsanity.com/paper/1307.0414/) (2013)
- [Unsupervised Machine Translation Using Monolingual Corpora Only](https://mlsanity.com/paper/1711.00043/) (2017)
- [Deep Learning for Text Style Transfer: A Survey](https://mlsanity.com/paper/2011.00416/) (2020)
- [A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings](https://mlsanity.com/paper/1805.06297/) (2018)
- [Unsupervised Alignment of Embeddings with Wasserstein Procrustes](https://mlsanity.com/paper/1805.11222/) (2018)
- [Large language models predict human sensory judgments across six modalities](https://mlsanity.com/paper/2302.01308/) (2023)
- [Gibbs Sampling with People](https://mlsanity.com/paper/2008.02595/) (2020)
- [Facial Emotion Recognition: State of the Art Performance on FER2013](https://mlsanity.com/paper/2105.03588/) (2021)
- [Words are all you need? Language as an approximation for human similarity judgments](https://mlsanity.com/paper/2206.04105/) (2022)
- [On the Informativeness of Supervision Signals](https://mlsanity.com/paper/2211.01407/) (2022)
- [A Discriminative Latent-Variable Model for Bilingual Lexicon Induction](https://mlsanity.com/paper/1808.09334/) (2018)
- [Exploring emotional prototypes in a high dimensional TTS latent space](https://mlsanity.com/paper/2105.01891/) (2021)
- [VoiceMe: Personalized voice generation in TTS](https://mlsanity.com/paper/2203.15379/) (2022)
- [Differences in Chinese and Western tourists faced with Japanese hospitality: A natural language processing approach](https://mlsanity.com/paper/2107.14681/) (2021)
- [Bridging the prosody GAP: Genetic Algorithm with People to efficiently sample emotional prosody](https://mlsanity.com/paper/2205.04820/) (2022)
- [Giving Robots a Voice: Human-in-the-Loop Voice Creation and open-ended Labeling](https://mlsanity.com/paper/2402.05206/) (2024)

## Built on by (most cited first, 3 of 3)

- [Show or Tell? Modeling the evolution of request-making in Human-LLM conversations](https://mlsanity.com/paper/2508.01213/) (2025)
- [Why Human Guidance Matters in Collaborative Vibe Coding](https://mlsanity.com/paper/2602.10473/) (2026)
- [Modeling Community Attitude through Reaction Tone: A Human-AI Collaborative Framework for Evaluating LLM Alignment with Linguistic Behaviors in Online Communities](https://mlsanity.com/paper/2605.27388/) (2026)

## Abstract (arXiv)

Conversational tones -- the manners and attitudes in which speakers communicate -- are essential to effective communication. Amidst the increasing popularization of Large Language Models (LLMs) over recent years, it becomes necessary to characterize the divergences in their conversational tones relative to humans. However, existing investigations of conversational modalities rely on pre-existing taxonomies or text corpora, which suffer from experimenter bias and may not be representative of real-world distributions for the studies' psycholinguistic domains. Inspired by methods from cognitive science, we propose an iterative method for simultaneously eliciting conversational tones and sentences, where participants alternate between two tasks: (1) one participant identifies the tone of a given sentence and (2) a different participant generates a sentence based on that tone. We run 100 iterations of this process with human participants and GPT-4, then obtain a dataset of sentences and frequent conversational tones. In an additional experiment, humans and GPT-4 annotated all sentences with all tones. With data from 1,339 human participants, 33,370 human judgments, and 29,900 GPT-4 queries, we show how our approach can be used to create an interpretable geometric representation of relations between conversational tones in humans and GPT-4. This work demonstrates how combining ideas from machine learning and cognitive science can address challenges in human-computer interactions.