Target-Independent Micro-Interventions for Predicting Training Response Across Language-Model Families
Not on the map. Why?
Abstract (arXiv)
Benchmark scores describe what a checkpoint can do now, but they do not determine how it will respond to the next training episode. We measure this missing state by branching four short, standardized, target-independent micro-interventions from the same checkpoint and recording their effects in a common capability space. Together with current capability, these responses form L-State; its pulse block supports a flexible direct readout and a structure-preserving operator readout. Under smooth local dynamics, the operator construction admits an end-to-end cross-family bound with explicit source- and target-family coordinate heterogeneity. In three-family leave-one-family-out development, both pulse readouts reduce source-standardized MSE by 39.4% relative to capability alone, while separating the best response and direction estimates. On sealed GLM-4-9B, the direct and operator readouts reduce MSE by 71.8% and 78.3%, respectively, and the operator readout raises sign balanced accuracy from 0.366 to 0.754. On sealed Granite-3.1-8B, the direct readout reaches RMSE 0.544 and a development-fitted action-wise selector reaches 0.554, compared with 1.172 for capability alone. A five-family audit finds that the operator coordinate varies by action and family, and that modeling these deviations improves retrospective held-trajectory prediction. Target-independent interventions therefore expose training-response information that current capability misses, with direct and structured readouts covering complementary transfer regimes.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 23 of its 26 references that are in the map. First posted on arXiv 8 Sep 2026.
Builds on
23 of 26 references.
References · 23
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2021Training Verifiers to Solve Math Word Problemscited by 9,743
- 2021Program Synthesis with Large Language Modelscited by 3,509
- 2016Pointer Sentinel Mixture Modelscited by 2,992
- 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
- 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
- 2022Editing Models with Task Arithmeticcited by 1,076
- 2020In Search of Lost Domain Generalizationcited by 1,042
- 2017Crowdsourcing Multiple Choice Science Questionscited by 742
- 2024LESS: Selecting Influential Data for Targeted Instruction Tuningcited by 514
- 20242 OLMo 2 Furiouscited by 302
- 2020LEEP: A New Measure to Evaluate Transferability of Learned Representationscited by 182
11 more
- 2024Observational Scaling Laws and the Predictability of Language Model Performancecited by 128
- 2022Training Trajectories of Language Models Across Scalescited by 80
- 2021Conditional probing: measuring usable information beyond a baselinecited by 61
- 2020Active Learning for Identification of Linear Dynamical Systemscited by 33
- 2024Selecting Large Language Model to Fine-tune via Rectified Scaling Lawcited by 33
- 2025LLM DNA: Tracing Model Evolution via Functional Representationscited by 15
- 2022Predicting Fine-Tuning Performance with Probingcited by 13
- 2024A Bayesian Model Selection Criterion for Selecting Pretraining Checkpointscited by 4
- 2026Transporting Task Vectors across Different Architectures without Trainingcited by 4
- 2026TuneAhead: Predicting Fine-tuning Performance Before Full Training Beginscited by 1
- 2026Training, learning and inference: unified dynamics of neural systemscited by 1
Built on by
Not cited yet.