mlsanityData to 7 Oct 2026

The One-Word Census: Answer-Choice Conformity Across 44 Language Models

arXiv 2607.12796 ↗First posted 14 Jul 2026cs.CL cs.AI cs.CYcited by 3

Not on the map. Why?

Abstract (arXiv)

When a language model must choose one answer from a large space of equally valid options, which answer does it choose, and how often is it the answer every other model chooses? Asked to "pick a word," 105 language models from more than twenty labs chose serendipity 46% of the time. We measure this convergence, and each model's share in it, with 96 single-turn prompts that each name a category with many valid one-word answers ("Name a tree."), asked eight times per model and scored by exact match, with no embeddings and no judge. A model's answer-choice surprisal is the average -log2 probability of its answers under the pooled answers of all other models. In 28 of 96 categories a single answer takes at least 80% of all answers. The concentration does not depend on small or persona-tuned models: the 87 major-lab models are at least as concentrated as the full field. Lightly post-trained and persona-tuned models are the most divergent; heavily post-trained assistants from the major labs are the most conformist. Models that avoid the modal answer mostly land on the same runner-up. Within the major providers' lineages, release order shows no panel-wide trend once model tier is controlled; GPT, Gemini, Grok and Qwen become more conformist across releases, and Claude's generation-5 releases reverse. On open checkpoints of three post-training pipelines, supervised fine-tuning is the largest step toward the field's answers. Against human category-production norms, the field is more concentrated than people in 18 of 20 shared categories. All prompts, transcripts, and code are public.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 26 of its 45 references that are in the map. First posted on arXiv 14 Jul 2026.

Builds on

26 of 45 references.

References · 26

  1. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,068
  2. 2023Whose Opinions Do Language Models Reflect?cited by 759
  3. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 598
  4. 2023LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Datasetcited by 449
  5. 2023Understanding the Effects of RLHF on LLM Generalisation and Diversitycited by 386
  6. 2023How is ChatGPT's behavior changing over time?cited by 265
  7. 2023Does Writing with Language Models Reduce Content Diversity?cited by 190
  8. 2023Generative artificial intelligence enhances creativity but reduces the diversity of novel contentcited by 165
  9. 2025Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)cited by 137
  10. 2020Evaluating the Evaluation of Diversity in Natural Language Generationcited by 127
  11. 2024Homogenization Effects of Large Language Models on Human Creative Ideationcited by 127
  12. 2024The Good, The Bad, and The Greedy: Evaluation of LLMs Should Not Ignore Non-Determinismcited by 113
14 more
  1. 2023The Curious Decline of Linguistic Diversity: Training Language Models on Synthetic Textcited by 97
  2. 2025Jointly Reinforcing Diversity and Quality in Language Model Generationscited by 79
  3. 2025NoveltyBench: Evaluating Language Models for Humanlike Diversitycited by 64
  4. 2024Forcing Diffuse Distributions out of Language Modelscited by 48
  5. 2025We're Different, We're the Same: Creative Homogeneity Across LLMscited by 41
  6. 2025KL-Regularized Reinforcement Learning is Designed to Mode Collapsecited by 29
  7. 2025LLM Probability Concentration: How Alignment Shrinks the Generative Horizoncited by 26
  8. 2025What and Whose Knowledge? Measuring Epistemic Diversity in Large Language Modelscited by 14
  9. 2026Where does output diversity collapse in post-training?cited by 10
  10. 2026One Token Is Enough: Fingerprinting and Verifying Large Language Models from Single-Token Output Distributionscited by 6
  11. 2026"Please, don't kill the only model that still feels human": Understanding the #Keep4o Backlashcited by 5
  12. 2026The Alignment Tax: Response Homogenization in Aligned LLMs and Its Implications for Uncertainty Estimationcited by 4
  13. 2026AI generates well-liked but templatic empathic responsescited by 3
  14. 2026Towards Physical Intuitions for Alignment Dynamics: A Case Study With Randomness Crystallizationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.