mlsanityData to 14 Oct 2026

Finite Certificates for In-Context Determinacy and a Threshold Theory of Emergence in Language Models

arXiv 2606.07623 ↗First posted 30 May 2026cs.LG cs.LOcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper develops a model-theoretic framework for verifying context-conditioned language-model behavior by replacing benchmark labels with finite semantic certificates. The first problem is finite determinacy: when do examples in a context force the answer to a query without changing model parameters? In finite-field linear task families, we prove an exact row-space criterion, compute the residual hypothesis count, derive full and query-local identification curves, and show that extracting a smallest forcing subcontext is NP-complete even for binary outputs. The second problem is threshold emergence: when does an apparent benchmark jump reflect a semantic transition rather than a discontinuity of the scoring map? We prove an anti-mirage theorem separating thresholded metrics from semantic confidence and give a rate-sensitive crossing bound for latent commitments becoming visible above threshold. The common semantic object is a confidence functional on definable events. We show that it is a Boolean probability measure, equivalently a Keisler measure on the relevant type space, whose measure-one formulas form a proper filter and whose Stone-space representation is invariant under definitional expansion. The resulting calculus provides finite context certificates, pair-separator hitting sets, query teaching dimension, prompt-preservation criteria, and scale-limit witnesses. Exact-arithmetic ancillary scripts reproduce the finite-field and threshold calculations and generate the data used by the figures.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 14 of its 34 references that are in the map. First posted on arXiv 30 May 2026.

Builds on

14 of 34 references.

References · 14

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2020Scaling Laws for Neural Language Modelscited by 6,586
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  5. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  6. 2022Emergent Abilities of Large Language Modelscited by 2,512
  7. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,135
  8. 2022In-context Learning and Induction Headscited by 964
  9. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 938
  10. 2022Transformers learn in-context by gradient descentcited by 732
  11. 2022What Can Transformers Learn In-Context? A Case Study of Simple Function Classescited by 715
  12. 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 684
2 more
  1. 2023Are Emergent Abilities of Large Language Models a Mirage?cited by 523
  2. 2023Transformers learn to implement preconditioned gradient descent for in-context learningcited by 279

Built on by

Not cited yet.