mlsanityData to 14 Oct 2026

Prompt-Model Interaction Reaches the Fixed Points: A deterministic, task-free structural readout -- and the factorizations of it that failed

arXiv 2608.21315 ↗First posted 21 Aug 2026cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

That a prompt's effect is not a property of the prompt is established: prompts optimised for one model degrade on another, and rankings reorder under neutral reformatting. That evidence is about task accuracy, which cannot say whether the interaction is a fact about task machinery or about the conditional distribution itself. We ask on a readout with no task in it: the fixed-point structure of the short-window argmax map x_{t+1} = argmax_x p(x | x_{t-1}, x_t), censused from 96 starts. It is deterministic, so nothing can be helped or hurt, and it exists only at short windows -- four of six models lose it entirely by window 16 -- so everything here concerns how a model reads a fragment. Two results. First, the interaction reaches this readout at full magnitude: nine tokens of conditioning move the fixed-point fraction across most of its range, change a four-way structural class, and reorder models, while instruction tuning worth 60.5 IFEval points moves the class by zero. Second, nothing we proposed carries it. Prefix length fails: the effect is not monotone. Four phenomenological factors -- prose-versus-markup, a universal direction, bidirectionality, instruct-resistance -- were each withdrawn within one run of being proposed, dissolved by widening the sample. And the nearest mechanistic account, attention-sink dominance of early tokens, predicts the sign of the shift on 2 of 5 models -- chance -- while a length-by-content cross shows it holds on real text and fails on our probe's uniformly random input, so we are outside its regime, not against it. One fixed nine-token prefix drives four models toward 0 and two toward 1; the bidirectionality survives in-distribution starts. On this readout the unit of explanation is the prompt-model pair. The recurring error it caught in us has a name: a criterion with a shape applied to a quantity with no room to vary.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 13 of its 17 references that are in the map. First posted on arXiv 21 Aug 2026.

Builds on

13 of 17 references.

References · 13

  1. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  2. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,135
  3. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 817
  4. 2024When Attention Sink Emerges in Language Models: An Empirical Viewcited by 208
  5. 2024When Benchmarks are Targets: Revealing the Sensitivity of Large Language Model Leaderboardscited by 137
  6. 2020A Theoretical Analysis of the Repetition Problem in Text Generationcited by 83
  7. 2024Mind Your Format: Towards Consistent Evaluation of In-Context Learning Improvementscited by 83
  8. 2024On the Worst Prompt Performance of Large Language Modelscited by 38
  9. 2024MAPO: Boosting Large Language Model Performance with Model-Adaptive Prompt Optimizationcited by 21
  10. 2025Unveiling Attractor Cycles in Large Language Models: A Dynamical Systems View of Successive Paraphrasingcited by 15
  11. 2025Repetitions are not all alike: distinct mechanisms sustain repetition in language modelscited by 10
  12. 2026One prompt is not enough: Instruction Sensitivity Undermines Embedding Model Evaluationcited by 3
1 more
  1. 2026What Iterated Self-Feeding Probes of Language Models Measure, and a test that separates the construction from the modelcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.