mlsanityData to 14 Oct 2026

THEIA: Learning Complete Kleene Three-Valued Logic in a Pure-Neural Modular Architecture

arXiv 2604.11284 ↗First posted 13 Apr 2026cs.LG cs.AI cs.LOcited by 0

On the map: Mechanistic interpretability → Steering and interpreting LLM reasoning

Abstract (arXiv)

We present THEIA, a 2.75M-parameter modular neural architecture that learns the complete Kleene three-valued logic (K3) truth table from task data without external symbolic inference or hand-encoded K3 gate primitives. Across 5 seeds it passes all 39 K3 rules at >99% per-rule accuracy. K3 learnability is not the central finding: Transformer baselines also pass all 39 rules, and flat MLPs match THEIA on Phase-1 accuracy within 0.04pp. The contributions are two properties of the learned system. (1) Uncertainty-verdict asymmetric propagation. THEIA preserves Has-Unknown at every upstream boundary (80.0/91.1/90.8/99.7% across Arith/Order/Set/Logic vs. ~52% majority) while final-verdict decodability stays at or below a 73.4% U-vs-non-U oracle reference under linear and nonlinear probes. Activation patching on non-absorbent T->U cases flips 4,898/4,898 OR and 4,719/4,719 AND pairs across 5 seeds, ruling out residual shortcuts. (2) Reliability spectrum under discretized end-to-end training, on tasks decomposable along the engine boundaries. A mod-3 sequential composition task generalizes from 5- to 500-step evaluation at 99.96+-0.04% (5 seeds). Under identical Gumbel-softmax training, flat MLPs collapse to chance by 50 steps; a 2x2 ResMLP grid reaches >=99% on only 3/20 (config, seed) trials; a pre-LN Transformer reaches 99.24+-0.34%. Straight-through discretization prevents 0.999^500 compounding; the architectural separator is sustaining Phase-1 accuracy under Phase-3 training, where flat MLPs fail. Auxiliary: under per-architecture development defaults (not optimizer-controlled), THEIA reaches 12/12 Kleene coverage 6.5x faster than a parameter-comparable 8L Transformer; this narrows to ~3.6x with Transformer-standard tuning and 4.93x with the same recipe on both. Ratios are config-specific, not asymptotic.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Steering and interpreting LLM reasoning, one of 13 directions in Mechanistic interpretability. Builds on 33 of its 49 references that are in the map. First posted on arXiv 13 Apr 2026.

Builds on

33 of 49 references.

References · 33

  1. 2017Attention Is All You Needcited by 60,361
  2. 2017Neural Message Passing for Quantum Chemistrycited by 3,487
  3. 2017Prototypical Networks for Few-shot Learningcited by 3,015
  4. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,739
  5. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  6. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
  7. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  8. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,148
  9. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,103
  10. 2018Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learningcited by 1,102
  11. 2020On Layer Normalization in the Transformer Architecturecited by 922
  12. 2023Progress measures for grokking via mechanistic interpretabilitycited by 839
21 more
  1. 2019Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological Viewcited by 545
  2. 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 503
  3. 2021Causal Abstractions of Neural Networkscited by 422
  4. 2015Neural Module Networkscited by 376
  5. 2017A Semantic Loss Function for Deep Learning with Symbolic Knowledgecited by 270
  6. 2022Exploring Length Generalization in Large Language Modelscited by 206
  7. 2019What graph neural networks cannot learn: depth vs widthcited by 201
  8. 2020Logic Tensor Networkscited by 199
  9. 2019What Can Neural Networks Reason About?cited by 159
  10. 2019Neural Logic Machinescited by 145
  11. 2020Logical Neural Networkscited by 119
  12. 2022The CLRS Algorithmic Reasoning Benchmarkcited by 106
  13. 2023NeurASP: Embracing Neural Networks into Answer Set Programmingcited by 92
  14. 2021From Statistical Relational to Neurosymbolic Artificial Intelligence: a Surveycited by 72
  15. 2018Neural Arithmetic Logic Unitscited by 70
  16. 2022Deep Differentiable Logic Gate Networkscited by 67
  17. 2023Scallop: A Language for Neurosymbolic Programmingcited by 63
  18. 2024Convolutional Differentiable Logic Gate Networkscited by 37
  19. 2024Discrete Neural Algorithmic Reasoningcited by 9
  20. 2019Neural Logic Networkscited by 7
  21. 2026Polynomial Surrogate Training for Differentiable Ternary Logic Gate Networkscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Categorical Construction of Logically Verifiable Neural Architectures
  2. 2026Distilling Formal Logic into Neural Spaces: A Kernel Alignment Approach for Signal Temporal Logic
  3. 2025Standard Neural Computation Alone Is Insufficient for Logical Intelligence
  4. 2025Constructing a Neuro-Symbolic Mathematician from First Principles
  5. 2026Differentiable Logic Synthesis: Spectral Coefficient Selection via Sinkhorn-Constrained Composition
  6. 2026NeuroNL2LTL: A Neurosymbolic Framework for Natural Language Translation of Linear Temporal Logic
  7. 2025Truth-Aware Decoding: A Program-Logic Approach to Factual Language Generation
  8. 2026Digital Metabolism: Decoupling Logic from Facts via Regenerative Unlearning -- Towards a Pure Neural Logic Core