# A Linguistic Comparison between Human and ChatGPT-Generated Conversations

arXiv: https://arxiv.org/abs/2401.16587 · HTML: https://mlsanity.com/paper/2401.16587/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 36 references that are in the map.
- First posted on arXiv 29 Jan 2024.

## Builds on (most cited first, 7 of 7)

- [The Science of Detecting LLM-Generated Texts](https://mlsanity.com/paper/2303.07205/) (2023)
- [Human heuristics for AI-generated language are flawed](https://mlsanity.com/paper/2206.07271/) (2022)
- [Can ChatGPT Defend its Belief in Truth? Evaluating LLM Reasoning via Debate](https://mlsanity.com/paper/2305.13160/) (2023)
- [Contrasting Linguistic Patterns in Human and LLM-Generated News Text](https://mlsanity.com/paper/2308.09067/) (2023)
- [Towards Possibilities & Impossibilities of AI-generated Text Detection: A Survey](https://mlsanity.com/paper/2310.15264/) (2023)
- [Visualizing Linguistic Diversity of Text Datasets Synthesized by Large Language Models](https://mlsanity.com/paper/2305.11364/) (2023)
- [Representing Affect Information in Word Embeddings](https://mlsanity.com/paper/2209.10583/) (2022)

## Built on by (most cited first, 9 of 9)

- [ChatSOP: An SOP-Guided MCTS Planning Framework for Controllable LLM Dialogue Agents](https://mlsanity.com/paper/2407.03884/) (2024)
- [Comparing LLM-generated and human-authored news text using formal syntactic theory](https://mlsanity.com/paper/2506.01407/) (2025)
- [Exploring Human-LLM Conversations: Mental Models and the Originator of Toxicity](https://mlsanity.com/paper/2407.05977/) (2024)
- [Increasing happiness through conversations with artificial intelligence](https://mlsanity.com/paper/2504.02091/) (2025)
- [Interpretable Stylistic Variation in Human and LLM Writing Across Genres, Models, and Decoding Strategies](https://mlsanity.com/paper/2604.14111/) (2026)
- [From Thinking to Output: Chain-of-Thought and Text Generation Characteristics in Reasoning Language Models](https://mlsanity.com/paper/2506.21609/) (2025)
- [Evaluating the Usage of African-American Vernacular English in Large Language Models](https://mlsanity.com/paper/2602.21485/) (2026)
- [Can Humans Detect AI? Mining Textual Signals of AI-Assisted Writing Under Varying Scrutiny Conditions](https://mlsanity.com/paper/2604.23471/) (2026)
- [The Dynamics of Human and AI-Generated Language: How Semantics Fluctuates across Different Timescales](https://mlsanity.com/paper/2606.11371/) (2026)

## Abstract (arXiv)

This study explores linguistic differences between human and LLM-generated dialogues, using 19.5K dialogues generated by ChatGPT-3.5 as a companion to the EmpathicDialogues dataset. The research employs Linguistic Inquiry and Word Count (LIWC) analysis, comparing ChatGPT-generated conversations with human conversations across 118 linguistic categories. Results show greater variability and authenticity in human dialogues, but ChatGPT excels in categories such as social processes, analytical style, cognition, attentional focus, and positive emotional tone, reinforcing recent findings of LLMs being "more human than human." However, no significant difference was found in positive or negative affect between ChatGPT and human dialogues. Classifier analysis of dialogue embeddings indicates implicit coding of the valence of affect despite no explicit mention of affect in the conversations. The research also contributes a novel, companion ChatGPT-generated dataset of conversations between two independent chatbots, which were designed to replicate a corpus of human conversations available for open access and used widely in AI research on language modeling. Our findings enhance understanding of ChatGPT's linguistic capabilities and inform ongoing efforts to distinguish between human and LLM-generated text, which is critical in detecting AI-generated fakes, misinformation, and disinformation.