Epistemic Throughput: Fundamental Limits of Attention-Constrained Inference
Not on the map. Why?
Abstract (arXiv)
Recent generative and tool-using AI systems can surface a large volume of candidates at low marginal cost, yet only a small fraction can be checked carefully. This creates a decoder-side bottleneck: downstream decision-makers must form reliable posteriors from many public records under scarce attention. We formalize this regime via Attention-Constrained Inference (ACI), in which a cheap screening stage processes K records and an expensive verification stage can follow up on at most B of them. Under Bayes log-loss, we study the maximum achievable reduction in posterior uncertainty per window, which we call epistemic throughput. Our main result is a ``JaKoB'' scaling law showing that epistemic throughput has a baseline term that grows linearly with verification and prevalence, and an additional information-leverage term that scales as √JKB, where J summarizes screening quality. Thus, expanding cheap screening can nonlinearly amplify scarce verification, even when informative records are rare. We further show that this scaling is tight in a weak-screening limit, and that in the sparse-verification regime (B ll K), substantial leverage requires heavy-tailed score distributions; for light-tailed scores the amplification is only logarithmic.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 9 of its 44 references that are in the map. First posted on arXiv 9 Feb 2026.
Builds on
9 of 44 references.
References · 9
- 2023GPT-4 Technical Reportcited by 18,132
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
- 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
- 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
- 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
- 2022MRKL Systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoningcited by 153
- 2015How much does your data exploration overfit? Controlling bias via information usagecited by 95
Built on by
Citing papers per year: 2026: 1.
1 paper.