Learn-by-Wire Training Control Governance: Bounded Autonomous Training Under Stress for Stability and Efficiency
Not on the map. Why?
Abstract (arXiv)
Modern language-model training is increasingly exposed to instability, degraded runs, and wasted compute, especially under aggressive learning-rate, scale, and runtime-stress conditions. This paper introduces Learn-by-Wire Guard (LBW-Guard), a bounded autonomous training-control governance layer that operates above AdamW. Rather than replacing the optimizer update rule, LBW-Guard observes training telemetry, interprets instability-sensitive regimes, and applies bounded control to optimizer execution while preserving fixed training objectives. We evaluate LBW-Guard in a Qwen2.5-centered stress-and-robustness suite using WikiText-103, with Qwen2.5-7B as the empirical anchor, model-size comparisons against Qwen2.5-3B and Qwen2.5-14B, learning-rate stress tests, gradient-clipping baselines, and a no-LoRA TinyLlama-1B full-parameter sanity check. In the 7B reference setting, LBW-Guard reduces final perplexity from 13.21 to 10.74, an 18.7% improvement, while reducing end-to-end time from 392.54s to 357.02s, a 1.10x speedup. Under stronger learning-rate stress, AdamW degrades to 1885.24 final perplexity at LR=3e-3 and 659.76 at LR=1e-3, whereas LBW-Guard remains trainable at 11.57 and 10.33, respectively. Gradient-clipping baselines do not reproduce this effect. These results support a scoped systems conclusion that stability-sensitive LLM training can benefit from a governance plane above the optimizer. LBW-Guard provides evidence that bounded runtime control can preserve productive compute under stress while remaining distinct from optimizer replacement and local gradient suppression.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 15 of its 20 references that are in the map. First posted on arXiv 18 May 2026.
Builds on
15 of 20 references.
References · 15
- 2020Scaling Laws for Neural Language Modelscited by 6,411
- 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
- 2022Training Compute-Optimal Large Language Modelscited by 2,927
- 2016Optimization Methods for Large-Scale Machine Learningcited by 1,140
- 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 896
- 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
- 2019On the Convergence of Adam and Beyondcited by 834
- 2021High-Performance Large-Scale Image Recognition Without Normalizationcited by 359
- 2020Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)cited by 318
- 2023A Theory on Adam Instability in Large-Scale Machine Learningcited by 54
- 2025Benchmarking Optimizers for Large Language Model Pretrainingcited by 53
- 2023Spike No More: Stabilizing the Pre-training of Large Language Modelscited by 51
3 more
Built on by
Not cited yet.