mlsanityData to 7 Oct 2026

Sixteen models, fewer than two voices: measuring ensemble dispersion where no answer is uniquely correct

arXiv 2608.00285 ↗First posted 31 Jul 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Sixteen language models drawn from ten families produced, on average, the semantic diversity of 1.69 distinct formulations of a psychotherapeutic case, against a single-model baseline of 1.43 from one model's own runs. Ensembles place more than one reading before a decision-maker on the premise that several models supply several perspectives. Dispersion over their outputs is measured both as diversity and as uncertainty, and both traditions validate it against a correctness criterion that this task does not admit. Measuring diversity is a solved problem: the Vendi Score, the exponential of the von Neumann entropy of a similarity matrix, is an effective number of distinct elements. What a single aggregate does not say is where the diversity comes from. We define a per-model dissent contribution, the complement of a model's mean similarity to the other members of its ensemble: a magnitude from the same matrix, not a decomposition of the spectral index, whose maximum identifies the most divergent voice. Crossing model and case, we test as a preregistered hypothesis whether model identity accounts for a non-zero share of the variance in dissent, and characterise the structure that test detects. The panel formulated fifteen stratified vignettes, yielding 7,082 formulations for analysis. Model identity was a detectable structuring factor of the dissent that remained, but the usual categories recovered it only partly: scale differences pointed in opposite directions across pairs, family grouped models on only five two-member lines, and the most divergent voice changed with panel composition, so that the surfaced outlier describes the ensemble rather than the model. Dissent did not track the interpretive openness for which the case bank was stratified; it was organised by clinical content instead, leaving the dispersion an ensemble produces a property to measure rather than assume.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 11 of its 30 references that are in the map. First posted on arXiv 31 Jul 2026.

Builds on

11 of 30 references.

References · 11

  1. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 795
  2. 2022The Vendi Score: A Diversity Evaluation Metric for Machine Learningcited by 299
  3. 2024Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similaritiescited by 156
  4. 2025Great Models Think Alike and this Undermines AI Oversightcited by 53
  5. 2025Correlated Errors in Large Language Modelscited by 42
  6. 2025We're Different, We're the Same: Creative Homogeneity Across LLMscited by 41
  7. 2026Where does output diversity collapse in post-training?cited by 10
  8. 2026Complementing Self-Consistency with Cross-Model Disagreement for Uncertainty Quantificationcited by 9
  9. 2026Diversity Collapse in Multi-Agent LLM Systems: Structural Coupling and Collective Failure in Open-Ended Idea Generationcited by 7
  10. 2026Are Diversity Metrics Measuring Diversity? A Capability-Controlled Audit of Majority-Vote Gain in LLM Ensemblescited by 7
  11. 2025The Vendiscope: An Algorithmic Microscope For Data Collectionscited by 5

Built on by

Not cited yet.