# The Collective Turing Test: Large Language Models Can Generate Realistic Multi-User Discussions

arXiv: https://arxiv.org/abs/2511.08592 · HTML: https://mlsanity.com/paper/2511.08592/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 62 references that are in the map.
- First posted on arXiv 29 Oct 2025.

## Builds on (most cited first, 22 of 22)

- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [Scaling Synthetic Data Creation with 1,000,000,000 Personas](https://mlsanity.com/paper/2406.20094/) (2024)
- [Social Simulacra: Creating Populated Prototypes for Social Computing Systems](https://mlsanity.com/paper/2208.04024/) (2022)
- [The Pushshift Reddit Dataset](https://mlsanity.com/paper/2001.08435/) (2020)
- [S3: Social-network Simulation System with Large Language Model-Empowered Agents](https://mlsanity.com/paper/2307.14984/) (2023)
- [OASIS: Open Agent Social Interaction Simulations with One Million Agents](https://mlsanity.com/paper/2411.11581/) (2024)
- [Simulating Social Media Using Large Language Models to Evaluate Alternative News Feed Algorithms](https://mlsanity.com/paper/2310.05984/) (2023)
- [POTATO: The Portable Text Annotation Tool](https://mlsanity.com/paper/2212.08620/) (2022)
- [Y Social: an LLM-powered Social Media Digital Twin](https://mlsanity.com/paper/2408.00818/) (2024)
- [Do Large Language Models Solve the Problems of Agent-Based Modeling? A Critical Review of Generative Social Simulations](https://mlsanity.com/paper/2504.03274/) (2025)
- [Disinformation and Social Bot Operations in the Run Up to the 2017 French Presidential Election](https://mlsanity.com/paper/1707.00086/) (2017)
- [Agent-Based Modelling Meets Generative AI in Social Network Simulations](https://mlsanity.com/paper/2411.16031/) (2024)
- [Emergence of Scale-Free Networks in Social Interactions among Large Language Models](https://mlsanity.com/paper/2312.06619/) (2023)
- [How malicious AI swarms can threaten democracy: The fusion of agentic AI and LLMs marks a new frontier in information warfare](https://mlsanity.com/paper/2506.06299/) (2025)
- [Quarantined! Examining the Effects of a Community-Wide Moderation Intervention on Reddit](https://mlsanity.com/paper/2009.11483/) (2020)
- [Do Platform Migrations Compromise Content Moderation? Evidence from r/The_Donald and r/Incels](https://mlsanity.com/paper/2010.10397/) (2020)
- [Online misinformation is linked to early COVID-19 vaccination hesitancy and refusal](https://mlsanity.com/paper/2104.10635/) (2021)
- [Make Reddit Great Again: Assessing Community Effects of Moderation Interventions on r/The_Donald](https://mlsanity.com/paper/2201.06455/) (2022)
- [Debunking in a World of Tribes](https://mlsanity.com/paper/1510.04267/) (2015)
- [A Turing Test: Are AI Chatbots Behaviorally Similar to Humans?](https://mlsanity.com/paper/2312.00798/) (2023)
- [Human Variability vs. Machine Consistency: A Linguistic Analysis of Texts Generated by Humans and Large Language Models](https://mlsanity.com/paper/2412.03025/) (2024)

## Built on by (most cited first, 2 of 2)

- [Computational Turing Test Reveals Systematic Differences Between Human and AI Language](https://mlsanity.com/paper/2511.04195/) (2025)
- [Illusory Truth or Mere Exposure? Model-Dependent Repetition Effects in LLM-Based Social Media Simulations](https://mlsanity.com/paper/2609.36278/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) offer new avenues to simulate online communities and social media. Potential applications range from testing the design of content recommendation algorithms to estimating the effects of content policies and interventions. However, the validity of using LLMs to simulate conversations between various users remains largely untested. We evaluated whether LLMs can convincingly mimic human group conversations on social media. We collected authentic human conversations from Reddit and generated artificial conversations on the same topic with two LLMs: Llama 3 70B and GPT-4o. When presented side-by-side to study participants, LLM-generated conversations were mistaken for human-created content 39% of the time. In particular, when evaluating conversations generated by Llama 3, participants correctly identified them as AI-generated only 56% of the time, barely better than random chance. Our study demonstrates that LLMs can generate social media conversations sufficiently realistic to deceive humans when reading them, highlighting both a promising potential for social simulation and a warning message about the potential misuse of LLMs to generate new inauthentic social media content.