mlsanityData to 14 Oct 2026

Reach Into The CHOIR: Free-List Elicitation Uncovers Distinct Model Voices in LLM Ensembles

arXiv 2609.38448 ↗First posted 29 Sep 2026cs.AI cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Open-ended LLM homogeneity can create false plurality when several systems appear to offer independent perspectives while returning the same familiar default. Single-pass answers obscure the distinction between agreement produced by a tightly constrained answer space, prompt-vocabulary echo, and broader answer spaces with stable alternatives beneath the surface. We introduce CHOIR (Collective Hierarchically-Ordered Inquiry Responses), a framework that adapts free-list elicitation from cognitive anthropology to LLM ensembles. CHOIR repeatedly elicits ranked lists, clusters items into prompt-level concepts, and measures concept salience across models, prompt variants, and persona conditions. We evaluate CHOIR on Infinity-Chat 100, an external prompt bank from recent work on open-ended model homogeneity, and on a 27-question targeted diagnostic bank designed to isolate mechanism-level contrasts. On Infinity-Chat 100, CHOIR reproduces high surface agreement (93/100 prompts above chance) while separating narrow prompts from broad prompts with recoverable depth. Across targeted probes and the external prompt bank, base-model identity remains the strongest recoverable signature, and persona prompts shift surfaced concepts within base-model signatures. A source-blind ranking module prioritises rare-but-stable candidates for later inspection. CHOIR turns open-ended homogeneity into a diagnostic measurement problem by asking where models converge, why they converge, and what remains reachable under structured depth probing.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 20 of its 31 references that are in the map. First posted on arXiv 29 Sep 2026.

Builds on

20 of 31 references.

References · 20

  1. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 752
  2. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 686
  3. 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 437
  4. 2024Self-Preference Bias in LLM-as-a-Judgecited by 258
  5. 2025Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)cited by 141
  6. 2025Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversitycited by 105
  7. 2025Evaluating the Diversity and Quality of LLM Generated Contentcited by 51
  8. 2023Diminished Diversity-of-Thought in a Standard Large Language Modelcited by 45
  9. 2024Evaluating Cultural Adaptability of a Large Language Model via Simulation of Synthetic Personascited by 37
  10. 2025The Personality Illusion: Revealing Dissociation Between Self-Reports & Behavior in LLMscited by 36
  11. 2025The Homogenizing Effect of Large Language Models on Human Expression and Thoughtcited by 35
  12. 2025Idiosyncrasies in Large Language Modelscited by 33
8 more
  1. 2025Multilingual Prompting for Improving LLM Generation Diversitycited by 25
  2. 2023How Far Can We Extract Diverse Perspectives from Large Language Models?cited by 22
  3. 2025Enhancing Jailbreak Attacks on LLMs via Persona Promptscited by 11
  4. 2025Deep Binding of Language Model Virtual Personas: a Study on Approximating Political Partisan Misperceptionscited by 9
  5. 2025Path Drift in Large Reasoning Models:How First-Person Commitments Override Safetycited by 5
  6. 2024The "LLM World of Words" English free association norms generated by large language modelscited by 4
  7. 2026Personality Expression Across Contexts: Linguistic and Behavioral Variation in LLM Agentscited by 4
  8. 2025Scaling Law in LLM Simulated Personality: More Detailed and Realistic Persona Profile Is All You Needcited by 3

Built on by

Not cited yet.