Modeling Next-Token Prediction as Left-Nested Intuitionistic Implication
Not on the map. Why?
Abstract (arXiv)
We introduce the Arrow Language Model, a neural architecture derived from an intuitionistic-logic interpretation of next-token prediction. Instead of representing tokens as additive embeddings mixed by attention, we encode a prefix as a left-nested implication chain whose structure preserves order through non-commutative composition. Next-token prediction corresponds to modus ponens, and sequence processing becomes constructive proof extension under the Curry--Howard correspondence. Our Prolog-based specialized theorem provers validate fundamental properties of the neural models, among which relations between commutative vs. non-commutative sequencing and single-token vs. multi-token prediction choices. We show that a neural architecture equivalent to multiplicative RNNs arises naturally from a proof-theoretic interpretation of next-token prediction as nested intuitionistic implication, we present a practical low-rank neural realization and position the model relative to Transformers and state-space models. Keywords: logic-based derivation of neural architectures, intuitionistic implicational logic, token-as-operator neural models, state-space models, alternatives to transformer-based foundational models.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 17 of its 24 references that are in the map. First posted on arXiv 7 Jan 2026.
Builds on
17 of 24 references.
References · 17
- 2017Attention Is All You Needcited by 60,361
- 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
- 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
- 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,850
- 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,336
- 2018Self-Attention with Relative Position Representationscited by 1,120
- 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,103
- 2016HyperNetworkscited by 1,027
- 2023Hyena Hierarchy: Towards Larger Convolutional Language Modelscited by 393
- 2023LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic Proverscited by 168
- 2023Coupling Large Language Models with Logic Programming for Robust and General Reasoning from Textcited by 59
5 more
- 2023Leveraging Large Language Models to Generate Answer Set Programscited by 33
- 2016On Multiplicative Integration with Recurrent Neural Networkscited by 15
- 2025Question Answering with LLMs and Learning from Answer Setscited by 4
- 2025A Solver-in-the-Loop Framework for Improving LLMs on Answer Set Programming for Logic Puzzle Solvingcited by 4
- 2023Full Automation of Goal-driven LLM Dialog Threads with And-Or Recursors and Refiner Oraclescited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2023Auto-Regressive Next-Token Predictors are Universal Learners
- 2025On the Limits of Hierarchically Embedded Logic in Classical Neural Networks
- 2026To See the Unseen: on the Generalization Ability of Transformers in Symbolic Reasoning
- 2025Emergent Symbolic Mechanisms Support Abstract Reasoning in Large Language Models
- 2026Pushing the Boundaries of Natural Reasoning: Interleaved Bonus from Formal-Logic Verification
- 2025Interpretable by AI Mother Tongue: Native Symbolic Reasoning in Neural Models
- 2025Tokenization Constraints in LLMs: A Study of Symbolic and Arithmetic Reasoning Limits
- 2025R2R: Efficiently Navigating Divergent Reasoning Paths with Small-Large Model Token Routing