mlsanityData to 14 Oct 2026

Structured Output Collapses Answer Diversity Across 44 Language Models

arXiv 2607.18476 ↗First posted 20 Jul 2026cs.CL cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

When a language model must choose one answer from a large space of equally valid options, a format clause -- "Reply with JSON only" -- changes which answer it chooses. We re-run the One-Word Census (arXiv:2607.12796): 31 wide-answer-space category prompts asked of 44 models, now with the reply requested in JSON -- no schema enforcement, no constrained decoding, only the request. Convergence deepens sharply: on the unconstrained "Pick a word" prompt the modal answer rises from 41% to 64% of the pool and distinct answers fall from 52 to 36; mean answer-choice surprisal drops from 1.80 to 1.58 bits. The tax is progressive: six of 44 models move individually (BH-FDR q=.10), all toward the mode, led by the most distinctive models, while the conformist floor is immobile. It is a sharpener, not a re-indexer -- the plain-chat modal answer survives in 28 of 31 categories. Defaults are register-indexed: a within-run re-sample (n=20) finds JSON shifts 53% of a model's stable chat defaults, mostly back to the crowd, and installs defaults absent from chat (Claude Fable 5 answers "cerulean" for colour 0% of the time in chat, 100% in JSON). Full-battery controls reveal a register gradient: compression is significant and specific to the answer-delivery formats models are trained to speak (JSON -0.22 bits, p=.0002; XML -0.19, p=.002), absent for YAML and CSV, and reversed for an arbitrary bracket wrapper (+0.13, p=.009) -- weighing the mechanism toward tool-use post-training. Enforcing the schema at the decoder (response_format) compresses no further than the request (-0.03 bits): the collapse lives in the model's response to the register, not the decoder. Structured output is how software consumes language models, and that surface is served by a measurably more homogeneous model than the chat surface on which models are evaluated, compared, and chosen.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 7 of its 9 references that are in the map. First posted on arXiv 20 Jul 2026.

Builds on

7 of 9 references.

References · 7

  1. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 817
  2. 2023Understanding the Effects of RLHF on LLM Generalisation and Diversitycited by 396
  3. 2025Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)cited by 141
  4. 2026The Constraint Tax: Measuring Validity-Correctness Tradeoffs in Structured Outputs for Small Language Modelscited by 4
  5. 2026Constraint Tax in Open-Weight LLMs: An Empirical Study of Tool Calling Suppression Under Structured Output Constraintscited by 3
  6. 2026The One-Word Census: Answer-Choice Conformity Across 105 Language Modelscited by 3
  7. 2025Automata-Based Steering of Large Language Models for Diverse Structured Generationcited by 2

Built on by

Not cited yet.