mlsanityData to 7 Oct 2026

Target-Independent Micro-Interventions for Predicting Training Response Across Language-Model Families

arXiv 2609.08618 ↗First posted 8 Sep 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Benchmark scores describe what a checkpoint can do now, but they do not determine how it will respond to the next training episode. We measure this missing state by branching four short, standardized, target-independent micro-interventions from the same checkpoint and recording their effects in a common capability space. Together with current capability, these responses form L-State; its pulse block supports a flexible direct readout and a structure-preserving operator readout. Under smooth local dynamics, the operator construction admits an end-to-end cross-family bound with explicit source- and target-family coordinate heterogeneity. In three-family leave-one-family-out development, both pulse readouts reduce source-standardized MSE by 39.4% relative to capability alone, while separating the best response and direction estimates. On sealed GLM-4-9B, the direct and operator readouts reduce MSE by 71.8% and 78.3%, respectively, and the operator readout raises sign balanced accuracy from 0.366 to 0.754. On sealed Granite-3.1-8B, the direct readout reaches RMSE 0.544 and a development-fitted action-wise selector reaches 0.554, compared with 1.172 for capability alone. A five-family audit finds that the operator coordinate varies by action and family, and that modeling these deviations improves retrospective held-trajectory prediction. Target-independent interventions therefore expose training-response information that current capability misses, with direct and structured readouts covering complementary transfer regimes.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 23 of its 26 references that are in the map. First posted on arXiv 8 Sep 2026.

Builds on

23 of 26 references.

References · 23

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2021Program Synthesis with Large Language Modelscited by 3,509
  4. 2016Pointer Sentinel Mixture Modelscited by 2,992
  5. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  6. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  7. 2022Editing Models with Task Arithmeticcited by 1,076
  8. 2020In Search of Lost Domain Generalizationcited by 1,042
  9. 2017Crowdsourcing Multiple Choice Science Questionscited by 742
  10. 2024LESS: Selecting Influential Data for Targeted Instruction Tuningcited by 514
  11. 20242 OLMo 2 Furiouscited by 302
  12. 2020LEEP: A New Measure to Evaluate Transferability of Learned Representationscited by 182
11 more
  1. 2024Observational Scaling Laws and the Predictability of Language Model Performancecited by 128
  2. 2022Training Trajectories of Language Models Across Scalescited by 80
  3. 2021Conditional probing: measuring usable information beyond a baselinecited by 61
  4. 2020Active Learning for Identification of Linear Dynamical Systemscited by 33
  5. 2024Selecting Large Language Model to Fine-tune via Rectified Scaling Lawcited by 33
  6. 2025LLM DNA: Tracing Model Evolution via Functional Representationscited by 15
  7. 2022Predicting Fine-Tuning Performance with Probingcited by 13
  8. 2024A Bayesian Model Selection Criterion for Selecting Pretraining Checkpointscited by 4
  9. 2026Transporting Task Vectors across Different Architectures without Trainingcited by 4
  10. 2026TuneAhead: Predicting Fine-tuning Performance Before Full Training Beginscited by 1
  11. 2026Training, learning and inference: unified dynamics of neural systemscited by 1

Built on by

Not cited yet.