# Readout Stability in Prefill-Only Decision Models:Zero-Label Prediction and Inference-Time Compute Allocation

arXiv: https://arxiv.org/abs/2610.07716 · HTML: https://mlsanity.com/paper/2610.07716/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 24 of its 69 references that are in the map.
- First posted on arXiv 6 Oct 2026.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 24 of 24)

- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance](https://mlsanity.com/paper/2305.05176/) (2023)
- [Reasoning Models Don't Always Say What They Think](https://mlsanity.com/paper/2505.05410/) (2025)
- [Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces](https://mlsanity.com/paper/1805.10190/) (2018)
- [Language Model Cascades](https://mlsanity.com/paper/2207.10342/) (2022)
- [A Unified Approach to Routing and Cascading for LLMs](https://mlsanity.com/paper/2410.10347/) (2024)
- [Learning to Route LLMs with Confidence Tokens](https://mlsanity.com/paper/2410.13284/) (2024)
- [Decision Theoretic Foundations for Conformal Prediction: Optimal Uncertainty Quantification for Risk-Averse Agents](https://mlsanity.com/paper/2502.02561/) (2025)
- [Prune 'n Predict: Optimizing LLM Decision-making with Conformal Prediction](https://mlsanity.com/paper/2501.00555/) (2024)
- [Fine-Tuning Large Language Models to Appropriately Abstain with Semantic Entropy](https://mlsanity.com/paper/2410.17234/) (2024)
- [Reasoning Models are Test Exploiters: Rethinking Multiple-Choice](https://mlsanity.com/paper/2507.15337/) (2025)
- [REFLEX with Jev for Efficient Selective Control in LLM Agents](https://mlsanity.com/paper/2609.26532/) (2026)
- [Decision Hijacking: Prompt Injection Attacks on Jev's Typed Probabilistic Decisions](https://mlsanity.com/paper/2609.28613/) (2026)
- [Replacing Large Language Models with Jev Decision Models for Low-Latency Edge Service Orchestration](https://mlsanity.com/paper/2609.22753/) (2026)
- [Evaluating and Benchmarking the System One Model Jev](https://mlsanity.com/paper/2609.37647/) (2026)
- [Just Ask Jev: Reinforcement Learning for Calibrated Decisions as a Zero-Shot Detector of AI Alignment Failures](https://mlsanity.com/paper/2609.29429/) (2026)
- [this-that-model-1.0: A typed decision model that decides in 30 ms, for a millionth of a cent](https://mlsanity.com/paper/2609.23886/) (2026)
- [Bridging the Knowledge-Prediction Gap in LLMs on Multiple-Choice Questions](https://mlsanity.com/paper/2509.23782/) (2025)
- [Explaining CLIP Zero-shot Predictions Through Concepts](https://mlsanity.com/paper/2603.28211/) (2026)
- [Localize-Then-Decide Guarantees for LLM Judgments](https://mlsanity.com/paper/2608.25824/) (2026)
- [Overwhelmed by Choice: Studying LLM Decision Making at Scale](https://mlsanity.com/paper/2609.32809/) (2026)

## Abstract (arXiv)

We present JevCascade, an inference-time compute allocation framework for prefill-only decision models, built on readout stability and zero-label restricted readout prediction. Prefill-only decision models score every candidate in a menu in a single forward pass and never decode, which makes one call one to two orders of magnitude cheaper than a same-scale generative language model. We show that this read-out structure carries a testable property. When an intervention changes only the menu and leaves the input text fixed, the post-intervention accuracy is already determined by the cached first-pass distribution. The restriction predictor restricts the pass-1 probabilities to the menu, renormalizes and reads off the argmax, with no labels and no second pass. Across six model families, ten core datasets (sixteen in total) and two task types, menu-only interventions are predicted to within 4.2 points, exactly for one family. The property is ordinal, not probabilistic, since a probability-level variant of the same estimator errs by 21.0 points, and class-specific, since same-scale generative models err by 1.6 to 15.8 points and degrade as they grow. It makes inference-time compute decidable before deployment. Uniform extra passes buy calibration but almost no accuracy; at matched cost a confidence cascade beats every scheme that re-asks the same model, and curating the menu beats enlarging the model, a 0.8B model on a curated 5-candidate menu reaching 95.4% on CLINC150 against 80.0% for a 4B model on the full 150-label menu.