mlsanityData to 7 Oct 2026

Learn-by-Wire Training Control Governance: Bounded Autonomous Training Under Stress for Stability and Efficiency

arXiv 2605.19008 ↗First posted 18 May 2026cs.AI cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Modern language-model training is increasingly exposed to instability, degraded runs, and wasted compute, especially under aggressive learning-rate, scale, and runtime-stress conditions. This paper introduces Learn-by-Wire Guard (LBW-Guard), a bounded autonomous training-control governance layer that operates above AdamW. Rather than replacing the optimizer update rule, LBW-Guard observes training telemetry, interprets instability-sensitive regimes, and applies bounded control to optimizer execution while preserving fixed training objectives. We evaluate LBW-Guard in a Qwen2.5-centered stress-and-robustness suite using WikiText-103, with Qwen2.5-7B as the empirical anchor, model-size comparisons against Qwen2.5-3B and Qwen2.5-14B, learning-rate stress tests, gradient-clipping baselines, and a no-LoRA TinyLlama-1B full-parameter sanity check. In the 7B reference setting, LBW-Guard reduces final perplexity from 13.21 to 10.74, an 18.7% improvement, while reducing end-to-end time from 392.54s to 357.02s, a 1.10x speedup. Under stronger learning-rate stress, AdamW degrades to 1885.24 final perplexity at LR=3e-3 and 659.76 at LR=1e-3, whereas LBW-Guard remains trainable at 11.57 and 10.33, respectively. Gradient-clipping baselines do not reproduce this effect. These results support a scoped systems conclusion that stability-sensitive LLM training can benefit from a governance plane above the optimizer. LBW-Guard provides evidence that bounded runtime control can preserve productive compute under stress while remaining distinct from optimizer replacement and local gradient suppression.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 15 of its 20 references that are in the map. First posted on arXiv 18 May 2026.

Builds on

15 of 20 references.

References · 15

  1. 2020Scaling Laws for Neural Language Modelscited by 6,411
  2. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  3. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  4. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,140
  5. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 896
  6. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  7. 2019On the Convergence of Adam and Beyondcited by 834
  8. 2021High-Performance Large-Scale Image Recognition Without Normalizationcited by 359
  9. 2020Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)cited by 318
  10. 2023A Theory on Adam Instability in Large-Scale Machine Learningcited by 54
  11. 2025Benchmarking Optimizers for Large Language Model Pretrainingcited by 53
  12. 2023Spike No More: Stabilizing the Pre-training of Large Language Modelscited by 51
3 more
  1. 2024The AdEMAMix Optimizer: Better, Faster, Oldercited by 47
  2. 2024Characterization of Large Language Model Development in the Datacentercited by 46
  3. 2025L4: Diagnosing Large-scale LLM Training Failures via Automated Log Analysiscited by 6

Built on by

Not cited yet.