The One-Word Census: Answer-Choice Conformity Across 44 Language Models
Not on the map. Why?
Abstract (arXiv)
When a language model must choose one answer from a large space of equally valid options, which answer does it choose, and how often is it the answer every other model chooses? Asked to "pick a word," 105 language models from more than twenty labs chose serendipity 46% of the time. We measure this convergence, and each model's share in it, with 96 single-turn prompts that each name a category with many valid one-word answers ("Name a tree."), asked eight times per model and scored by exact match, with no embeddings and no judge. A model's answer-choice surprisal is the average -log2 probability of its answers under the pooled answers of all other models. In 28 of 96 categories a single answer takes at least 80% of all answers. The concentration does not depend on small or persona-tuned models: the 87 major-lab models are at least as concentrated as the full field. Lightly post-trained and persona-tuned models are the most divergent; heavily post-trained assistants from the major labs are the most conformist. Models that avoid the modal answer mostly land on the same runner-up. Within the major providers' lineages, release order shows no panel-wide trend once model tier is controlled; GPT, Gemini, Grok and Qwen become more conformist across releases, and Claude's generation-5 releases reverse. On open checkpoints of three post-training pipelines, supervised fine-tuning is the largest step toward the field's answers. Against human category-production norms, the field is more concentrated than people in 18 of 20 shared categories. All prompts, transcripts, and code are public.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 26 of its 45 references that are in the map. First posted on arXiv 14 Jul 2026.
Builds on
26 of 45 references.
References · 26
- 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,068
- 2023Whose Opinions Do Language Models Reflect?cited by 759
- 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 598
- 2023LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Datasetcited by 449
- 2023Understanding the Effects of RLHF on LLM Generalisation and Diversitycited by 386
- 2023How is ChatGPT's behavior changing over time?cited by 265
- 2023Does Writing with Language Models Reduce Content Diversity?cited by 190
- 2023Generative artificial intelligence enhances creativity but reduces the diversity of novel contentcited by 165
- 2025Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)cited by 137
- 2020Evaluating the Evaluation of Diversity in Natural Language Generationcited by 127
- 2024Homogenization Effects of Large Language Models on Human Creative Ideationcited by 127
- 2024The Good, The Bad, and The Greedy: Evaluation of LLMs Should Not Ignore Non-Determinismcited by 113
14 more
- 2023The Curious Decline of Linguistic Diversity: Training Language Models on Synthetic Textcited by 97
- 2025Jointly Reinforcing Diversity and Quality in Language Model Generationscited by 79
- 2025NoveltyBench: Evaluating Language Models for Humanlike Diversitycited by 64
- 2024Forcing Diffuse Distributions out of Language Modelscited by 48
- 2025We're Different, We're the Same: Creative Homogeneity Across LLMscited by 41
- 2025KL-Regularized Reinforcement Learning is Designed to Mode Collapsecited by 29
- 2025LLM Probability Concentration: How Alignment Shrinks the Generative Horizoncited by 26
- 2025What and Whose Knowledge? Measuring Epistemic Diversity in Large Language Modelscited by 14
- 2026Where does output diversity collapse in post-training?cited by 10
- 2026One Token Is Enough: Fingerprinting and Verifying Large Language Models from Single-Token Output Distributionscited by 6
- 2026"Please, don't kill the only model that still feels human": Understanding the #Keep4o Backlashcited by 5
- 2026The Alignment Tax: Response Homogenization in Aligned LLMs and Its Implications for Uncertainty Estimationcited by 4
- 2026AI generates well-liked but templatic empathic responsescited by 3
- 2026Towards Physical Intuitions for Alignment Dynamics: A Case Study With Randomness Crystallizationcited by 1
Built on by
Citing papers per year: 2026: 3.
3 papers.