Identifiability and Order-Dimension Limits of In-Context Learning on Partial Orders
Not on the map. Why?
Abstract (arXiv)
In-context learning is commonly formalized as inference from examples of a function. Partial orders instead combine transitivity, antisymmetry, and incomparability, so a finite prompt may not determine a queried comparison. We develop a theory of in-context learning on partial orders that separates logical identifiability, prompt teaching cost, structural complexity, and the exact capacity of a formal coordinate-decoder class. A version-space semantics makes background knowledge and open- versus closed-world assumptions explicit. For finite open-world prompts with positive and negative comparisons, we prove an exact completion trichotomy: after taking the reflexive transitive closure of the positive demonstrations, a query is forced true, forced false because every true completion creates a cycle or violates a negative demonstration, or remains genuinely ambiguous. For a known n-element universe, we characterize the open-world teaching number as the number of covers plus a blocker-set hitting number, prove that its maximum over all n-element posets is n(n-1) and is uniquely attained by the antichain, and identify the blocker term as the exact cost of open-world rather than complete-Hasse semantics. We formalize prompt-dependent s-coordinate decoders and use the classical coordinate-order equivalence to obtain an exact representation boundary: dimension at most s is necessary and sufficient, while width at most s is a convenient sufficient condition.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 16 of its 24 references that are in the map. First posted on arXiv 14 Aug 2026.
Builds on
16 of 24 references.
References · 16
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2022What Can Transformers Learn In-Context? A Case Study of Simple Function Classescited by 700
- 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 671
- 2023In-Context Learning Creates Task Vectorscited by 302
- 2023Can Language Models Solve Graph Problems in Natural Language?cited by 249
- 2023How Do Transformers Learn In-Context Beyond Simple Functions? A Case Study on Learning with Representationscited by 72
- 2023Understanding In-Context Learning in Transformers and LLMs by Learning to Learn Discrete Functionscited by 62
- 2025Which Attention Heads Matter for In-Context Learning?cited by 61
- 2024Transformers are Universal In-context Learnerscited by 42
- 2024An Information-Theoretic Analysis of In-Context Learningcited by 39
- 2025Large Language Models are Good Relational Learnerscited by 22
- 2025Depth-Width tradeoffs in Algorithmic Reasoning of Graph Tasks with Transformerscited by 11
4 more
- 2024Ask, and it shall be given: On the Turing completeness of promptingcited by 10
- 2026OpenRFM: Dissecting Relational In-Context Learningcited by 4
- 2026In-Context Learning Operates as Concept Subspace Learningcited by 3
- 2025Assessing the Limits of In-Context Learning beyond Functions using Partially Ordered Relationcited by 1
Built on by
Not cited yet.