# Exploring Human-LLM Conversations: Mental Models and the Originator of Toxicity

arXiv: https://arxiv.org/abs/2407.05977 · HTML: https://mlsanity.com/paper/2407.05977/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 56 references that are in the map.
- First posted on arXiv 8 Jul 2024.

## Builds on (most cited first, 22 of 22)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [OpenAssistant Conversations -- Democratizing Large Language Model Alignment](https://mlsanity.com/paper/2304.07327/) (2023)
- [WildChat: 1M ChatGPT Interaction Logs in the Wild](https://mlsanity.com/paper/2405.01470/) (2024)
- [LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset](https://mlsanity.com/paper/2309.11998/) (2023)
- [The Design and Implementation of XiaoIce, an Empathetic Social Chatbot](https://mlsanity.com/paper/1812.08989/) (2018)
- [Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directions](https://mlsanity.com/paper/2310.19775/) (2023)
- [A Computational Approach to Politeness with Application to Social Factors](https://mlsanity.com/paper/1306.6078/) (2013)
- [Deceiving Google's Perspective API Built for Detecting Toxic Comments](https://mlsanity.com/paper/1702.08138/) (2017)
- [Datasets for Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2402.18041/) (2024)
- ["It's Weird That it Knows What I Want": Usability and Interactions with Copilot for Novice Programmers](https://mlsanity.com/paper/2304.02491/) (2023)
- [Handling Bias in Toxic Speech Detection: A Survey](https://mlsanity.com/paper/2202.00126/) (2022)
- [Determinants of LLM-assisted Decision-Making](https://mlsanity.com/paper/2402.17385/) (2024)
- [People cannot distinguish GPT-4 from a human in a Turing test](https://mlsanity.com/paper/2405.08007/) (2024)
- [The Shifted and The Overlooked: A Task-oriented Investigation of User-GPT Interactions](https://mlsanity.com/paper/2310.12418/) (2023)
- [On the Challenges of Using Black-Box APIs for Toxicity Evaluation in Research](https://mlsanity.com/paper/2304.12397/) (2023)
- [A Taxonomy for Human-LLM Interaction Modes: An Initial Exploration](https://mlsanity.com/paper/2404.00405/) (2024)
- [Impact of Guidance and Interaction Strategies for LLM Use on Learner Performance and Perception](https://mlsanity.com/paper/2310.13712/) (2023)
- [Does ChatGPT have Theory of Mind?](https://mlsanity.com/paper/2305.14020/) (2023)
- [Negotiating with LLMS: Prompt Hacks, Skill Gaps, and Reasoning Deficits](https://mlsanity.com/paper/2312.03720/) (2023)
- [Investigating Agency of LLMs in Human-AI Collaboration Tasks](https://mlsanity.com/paper/2305.12815/) (2023)
- [An HCI-Centric Survey and Taxonomy of Human-Generative-AI Interactions](https://mlsanity.com/paper/2310.07127/) (2023)
- [A Linguistic Comparison between Human and ChatGPT-Generated Conversations](https://mlsanity.com/paper/2401.16587/) (2024)

## Built on by (most cited first, 4 of 4)

- [Acceptable Use Policies for Foundation Models](https://mlsanity.com/paper/2409.09041/) (2024)
- [Consistency of Responses and Continuations Generated by Large Language Models on Social Media](https://mlsanity.com/paper/2501.08102/) (2025)
- [OpenEthics: A Comprehensive Ethical Evaluation of Open-Source Generative Large Language Models](https://mlsanity.com/paper/2505.16036/) (2025)
- [The Limits of Goal-Setting Theory in LLM-Driven Assessment](https://mlsanity.com/paper/2510.06997/) (2025)

## Abstract (arXiv)

This study explores real-world human interactions with large language models (LLMs) in diverse, unconstrained settings in contrast to most prior research focusing on ethically trimmed models like ChatGPT for specific tasks. We aim to understand the originator of toxicity. Our findings show that although LLMs are rightfully accused of providing toxic content, it is mostly demanded or at least provoked by humans who actively seek such content. Our manual analysis of hundreds of conversations judged as toxic by APIs commercial vendors, also raises questions with respect to current practices of what user requests are refused to answer. Furthermore, we conjecture based on multiple empirical indicators that humans exhibit a change of their mental model, switching from the mindset of interacting with a machine more towards interacting with a human.