# Predicting LLM Reasoning Performance with Small Proxy Model

arXiv: https://arxiv.org/abs/2509.21013 · HTML: https://mlsanity.com/paper/2509.21013/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 15 of its 52 references that are in the map.
- First posted on arXiv 25 Sep 2025.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 15 of 15)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [DeepSeek-V3 Technical Report](https://mlsanity.com/paper/2412.19437/) (2024)
- [The Falcon Series of Open Language Models](https://mlsanity.com/paper/2311.16867/) (2023)
- [2 OLMo 2 Furious](https://mlsanity.com/paper/2501.00656/) (2024)
- [The rising costs of training frontier AI models](https://mlsanity.com/paper/2405.21015/) (2024)
- [MiniCPM4: Ultra-Efficient LLMs on End Devices](https://mlsanity.com/paper/2506.07900/) (2025)
- [Densing Law of LLMs](https://mlsanity.com/paper/2412.04315/) (2024)
- [100K or 100 Days: Trade-offs when Pre-Training with Academic Resources](https://mlsanity.com/paper/2410.23261/) (2024)
- [Trillion 7B Technical Report](https://mlsanity.com/paper/2504.15431/) (2025)
- [Predictable Emergent Abilities of LLMs: Proxy Tasks Are All You Need](https://mlsanity.com/paper/2412.07111/) (2024)
- [The Hidden Costs of AI: A Review of Energy, E-Waste, and Inequality in Model Development](https://mlsanity.com/paper/2507.09611/) (2025)

## Built on by (most cited first, 5 of 5)

- [You Don't Need to Run Every Eval](https://mlsanity.com/paper/2606.24020/) (2026)
- [Fast and Accurate Probing of In-Training LLMs' Downstream Performances](https://mlsanity.com/paper/2604.01025/) (2026)
- [SuperValid: Capability-Aligned OOD Validation for Generalizable Downstream Scaling](https://mlsanity.com/paper/2605.28179/) (2026)
- [Validity Threats for Foundation Model Research](https://mlsanity.com/paper/2606.05029/) (2026)
- [Surrogate Fidelity: When Can Open LLMs Explain Closed Ones?](https://mlsanity.com/paper/2606.32008/) (2026)

## Abstract (arXiv)

Given the prohibitive cost of pre-training large language models, it is essential to leverage smaller proxy models to optimize datasets before scaling up. However, this approach becomes challenging for reasoning capabilities, which exhibit emergent behavior that only appear reliably at larger model sizes, often exceeding 7B parameters. To address this, we introduce rBridge, showing that small proxies (≤1B) can effectively predict large-model reasoning by aligning more closely with (1) the pre-training objective and (2) the target task. rBridge achieves this by weighting negative log-likelihood with task alignment, using reasoning traces from frontier models as gold labels. In our experiments, rBridge (i) reduces dataset ranking costs by over 100x relative to the best baseline, (ii) achieves the strongest correlation across six reasoning benchmarks at 1B to 32B scale, and (iii) zero-shot transfers predictive relationships across pre-training datasets at 1B to 7B scale. These findings indicate that rBridge offers a practical path for exploring reasoning-oriented pre-training at lower cost.