A Frozen 12B Beats Frontier Models on Verified Work: 100% Accuracy, 0 Tokens, Bit-Exact, Forever
Not on the map. Why?
Abstract (arXiv)
Improving a language model today means retraining it: enormous compute, a new opaque model each cycle, non-deterministic output. We take the opposite path: the model stays frozen, and a persistent memory of verified solutions grows beside it. Once a problem family is solved and has passed an independent verification step that never consults the answer key, every new instance of that family is answered at zero generation tokens, bit-exact, deterministically. Across 180 fresh instances spanning nine problem families, four architectures from four vendors - dense and mixture-of-experts - each score 180/180 at zero generation tokens per answer: execution-bound capability decoupled from parameter scaling. A negative control attributes the capability fully to the memory: emptied, it solves nothing. The same verify-before-store contract holds for open-ended reasoning: 88/88 consistency-gated acceptances across all four models, machine-checked formal proof, and reasoning-method transfer at 77/80. Memory selection takes 1.4 microseconds; a full reuse completes in 6-23 ms at 36 mWh. Approximate similarity retrieval selects the wrong item 94.3% of the time on a 4,500-item verified store where exact addressing makes zero errors. The store also serves as working context at a scale no shipped engine matches: a 6,000,000-token movable window on a single 46 GB GPU at flat memory, where vLLM stops at 30,399 tokens and SGLang silently truncates past 32,000. On published benchmarks, frontier models remain far ahead of any 12B at raw from-scratch reasoning; on everything this system has solved and verified, the comparison inverts: a frontier API call pays a fresh generation pass on every query, forever, while verified reuse costs zero tokens and returns the identical bits every time. A public testbench with free, rate-limited access accompanies this report: https://corbenic-galahad-bench.hf.space
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 18 of its 29 references that are in the map. First posted on arXiv 26 Jul 2026.
Builds on
18 of 29 references.
References · 18
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
- 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
- 2023LeanDojo: Theorem Proving with Retrieval-Augmented Language Modelscited by 427
- 2022CodeT: Code Generation with Generated Testscited by 384
- 2023Large Language Models as Tool Makerscited by 303
- 2023Prompt Cache: Modular Attention Reuse for Low-Latency Inferencecited by 268
- 2026Gemma 4 Technical Reportcited by 259
- 2024CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusioncited by 223
- 2025Memp: Exploring Agent Procedural Memorycited by 103
- 2024TroVE: Inducing Verifiable and Efficient Toolboxes for Solving Programmatic Taskscited by 75
- 2025Agentic Plan Caching: Test-Time Memory for Fast and Cost-Efficient LLM Agentscited by 27
6 more
- 2026From Lossy to Verified: A Provenance-Aware Tiered Memory for Agentscited by 17
- 2026VeriCache: Turning Lossy KV Cache into Lossless LLM Inferencecited by 5
- 2026Byte-Exact Deduplication in Retrieval-Augmented Generation: A Three-Regime Empirical Analysis Across Public Benchmarkscited by 4
- 2026Portable Agent Memory: A Protocol for Cryptographically-Verified Memory Transfer Across Heterogeneous AI Agentscited by 4
- 2026Merlin: Deterministic Byte-Exact Deduplication for Lossless Context Optimization in Large Language Model Inferencecited by 2
- 2026Trained Persistent Memory for Frozen Decoder-Only LLMscited by 1
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025Variation in Verification: Understanding Verification Dynamics in Large Language Models
- 2026Divergent-Convergent Reasoning: Scaling Test-Time Compute through Structured Solution Synthesis
- 2025Beyond Exponential Decay: Rethinking Error Accumulation in Large Language Models
- 2026Proof-Carrying Cognition: Closing the Verification Gap with Reality-Settled Reward
- 2025BeyondBench: Contamination-Resistant Evaluation of Reasoning in Language Models
- 2025T1: Tool-integrated Verification for Test-time Compute Scaling in Small Language Models
- 2026Empirical Evidence of Complexity-Induced Limits in Large Language Models on Finite Discrete State-Space Problems with Explicit Validity Constraints
- 2026Dissecting Failure Dynamics in Large Language Model Reasoning