# Training Under Challenge: Executable Certificates and Challenge-Closed Optimality for Neural Networks

arXiv: https://arxiv.org/abs/2608.12655 · HTML: https://mlsanity.com/paper/2608.12655/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 79 references that are in the map.
- First posted on arXiv 12 Aug 2026.

## Builds on (most cited first, 5 of 5)

- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [PACT: Parameterized Clipping Activation for Quantized Neural Networks](https://mlsanity.com/paper/1805.06085/) (2018)
- [Closed-Form Last Layer Optimization](https://mlsanity.com/paper/2510.04606/) (2025)
- [Tight Worst-Case Bounds for the Smallest Eigenvalue of ReLU NTK Gram Matrices](https://mlsanity.com/paper/2608.03368/) (2026)
- [Trust, but Verify: Peeling Low-Bit Transformer Networks for Training Monitoring](https://mlsanity.com/paper/2605.02853/) (2026)

## Abstract (arXiv)

A flat training curve does not reveal whether a neural network has reached a global optimum, is locally trapped, is representation-limited, or is mismatched to its trainer. We introduce Training Under Challenge, an executable-certificate framework in which predeclared, architecture-valid procedures construct complete alternatives in the same certified class and reevaluate the same objective. Any lower-valued candidate is a replayable witness that lower-bounds the checkpoint's empirical global-optimality gap. Passing a finite suite is only suite-relative; global-gap conclusions require a separately justified coverage mechanism. We define a resource-indexed challenge-power modulus that characterizes the largest gap compatible with passage. For squared loss, current block-decrease operators make coverage checkable and yield uniform and realized-residual bounds. We prove the converse frontier: without coverage, a first-order ReLU trainer can reach infinitely many exact conditional head optima while converging to a non-global point. On a channel-gated ResNet-18 distillation problem with known optimum, eight internal challenges cover all 240 audited output directions, and realized-residual bounds lie within factors of 1.74--3.02 of the true gap. Paired predictive certificates separate decoder under-use from representation insufficiency, while quantized-denoising studies demonstrate diagnosis, repair, and current-state recertification.