# Small Models Scout Bottleneck Order for Large-Model Data Control

arXiv: https://arxiv.org/abs/2608.14936 · HTML: https://mlsanity.com/paper/2608.14936/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 13 references that are in the map.
- First posted on arXiv 14 Aug 2026.

## Builds on (most cited first, 13 of 13)

- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks](https://mlsanity.com/paper/1502.05698/) (2015)
- [RegMix: Data Mixture as Regression for Language Model Pre-training](https://mlsanity.com/paper/2407.01492/) (2024)
- [Skill-it! A Data-Driven Skills Framework for Understanding and Training Language Models](https://mlsanity.com/paper/2307.14430/) (2023)
- [SmallToLarge (S2L): Scalable Data Selection for Fine-tuning Large Language Models by Summarizing Training Trajectories of Small Models](https://mlsanity.com/paper/2403.07384/) (2024)
- [Aioli: A Unified Optimization Framework for Language Model Data Mixing](https://mlsanity.com/paper/2411.05735/) (2024)
- [Data Mixing Can Induce Phase Transitions in Knowledge Acquisition](https://mlsanity.com/paper/2505.18091/) (2025)
- [What do Language Models Learn and When? The Implicit Curriculum Hypothesis](https://mlsanity.com/paper/2604.08510/) (2026)
- [TiKMiX: Take Data Influence into Dynamic Mixture for Language Model Pre-training](https://mlsanity.com/paper/2508.17677/) (2025)
- [Curriculum Learning for LLM Pretraining: An Analysis of Learning Dynamics](https://mlsanity.com/paper/2601.21698/) (2026)
- [Dynamic Proxy-Mixing: Transferring Replay Controllers from Small to Large Models for Continual Instruction Tuning](https://mlsanity.com/paper/2606.00400/) (2026)
- [AC-ODM: Actor--Critic Online Data Mixing for Sample-Efficient LLM Pretraining](https://mlsanity.com/paper/2505.23878/) (2025)
- [RegMix-D: Dynamic Data Mixing via Proxy Training Trajectories](https://mlsanity.com/paper/2606.18663/) (2026)

## Abstract (arXiv)

Small proxy models are commonly used to identify data mixtures for larger-scale training. We ask whether their training trajectories reveal another transferable structure: the order in which larger models should resolve skill bottlenecks. We formulate first-passage skill training, where each monitored skill has a target floor and the objective is to minimize the tokens required to reach all floors. We introduce LogFloor, a closed-loop controller that directs each round toward current bottlenecks, producing phase-ordered resolution trajectories. Across five bAbI skill slices on Qwen2.5-1.5B, LogFloor reduces token cost by 56.2% on average. In 70M-to-12B transfer, three-round replay of a 70M scout path reaches every floor in all eight target runs, saving 30.9% by pair mean, 39.4% in pooled training tokens, and 37.6% under source-cost accounting. On MMLU-control, a frozen scout path succeeds across all eight 12B runs. Collapsing a path to its static marginal mixture or reversing its phase order removes most benefits, while bottleneck labels alone remain partially useful. These results identify phase-ordered bottleneck resolution as a transferable curriculum structure for monitored skill-targeted training.