Finite Certificates for In-Context Determinacy and a Threshold Theory of Emergence in Language Models
Not on the map. Why?
Abstract (arXiv)
This paper develops a model-theoretic framework for verifying context-conditioned language-model behavior by replacing benchmark labels with finite semantic certificates. The first problem is finite determinacy: when do examples in a context force the answer to a query without changing model parameters? In finite-field linear task families, we prove an exact row-space criterion, compute the residual hypothesis count, derive full and query-local identification curves, and show that extracting a smallest forcing subcontext is NP-complete even for binary outputs. The second problem is threshold emergence: when does an apparent benchmark jump reflect a semantic transition rather than a discontinuity of the scoring map? We prove an anti-mirage theorem separating thresholded metrics from semantic confidence and give a rate-sensitive crossing bound for latent commitments becoming visible above threshold. The common semantic object is a confidence functional on definable events. We show that it is a Boolean probability measure, equivalently a Keisler measure on the relevant type space, whose measure-one formulas form a proper filter and whose Stone-space representation is invariant under definitional expansion. The resulting calculus provides finite context certificates, pair-separator hitting sets, query teaching dimension, prompt-preservation criteria, and scale-limit witnesses. Exact-arithmetic ancillary scripts reproduce the finite-field and threshold calculations and generate the data used by the figures.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 14 of its 34 references that are in the map. First posted on arXiv 30 May 2026.
Builds on
14 of 34 references.
References · 14
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2023GPT-4 Technical Reportcited by 18,396
- 2020Scaling Laws for Neural Language Modelscited by 6,586
- 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
- 2022Training Compute-Optimal Large Language Modelscited by 3,019
- 2022Emergent Abilities of Large Language Modelscited by 2,512
- 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,135
- 2022In-context Learning and Induction Headscited by 964
- 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 938
- 2022Transformers learn in-context by gradient descentcited by 732
- 2022What Can Transformers Learn In-Context? A Case Study of Simple Function Classescited by 715
- 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 684
2 more
Built on by
Not cited yet.