Empirical Validation of the Classification-Verification Dichotomy for AI Safety Gates
Not on the map. Why?
Abstract (arXiv)
Can classifier-based safety gates maintain reliable oversight as AI systems improve over hundreds of iterations? We provide comprehensive empirical evidence that they cannot. On a self-improving neural controller (d=240), eighteen classifier configurations -- spanning MLPs, SVMs, random forests, k-NN, Bayesian classifiers, and deep networks -- all fail the dual conditions for safe self-improvement. Three safe RL baselines (CPO, Lyapunov, safety shielding) also fail. Results extend to MuJoCo benchmarks (Reacher-v4 d=496, Swimmer-v4 d=1408, HalfCheetah-v4 d=1824). At controlled distribution separations up to delta_s=2.0, all classifiers still fail -- including the NP-optimal test and MLPs with 100% training accuracy -- demonstrating structural impossibility. We then show the impossibility is specific to classification, not to safe self-improvement itself. A Lipschitz ball verifier achieves zero false accepts across dimensions d in {84, 240, 768, 2688, 5760, 9984, 17408} using provable analytical bounds (unconditional delta=0). Ball chaining enables unbounded parameter-space traversal: on MuJoCo Reacher-v4, 10 chains yield +4.31 reward improvement with delta=0; on Qwen2.5-7B-Instruct during LoRA fine-tuning, 42 chain transitions traverse 234x the single-ball radius with zero safety violations across 200 steps. A 50-prompt oracle confirms oracle-agnosticity. Compositional per-group verification enables radii up to 37x larger than full-network balls. At d<=17408, delta=0 is unconditional; at LLM scale, conditional on estimated Lipschitz constants.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 14 of its 31 references that are in the map. First posted on arXiv 31 Mar 2026.
Builds on
14 of 31 references.
References · 14
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
- 2016Concrete Problems in AI Safetycited by 1,642
- 2017Constrained Policy Optimizationcited by 983
- 2017Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networkscited by 744
- 2017Provable defenses against adversarial examples via the convex outer adversarial polytopecited by 567
- 2017Safe Reinforcement Learning via Shieldingcited by 449
- 2018Efficient Neural Network Robustness Certification with General Activation Functionscited by 445
- 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 366
- 2018Towards Fast Computation of Certified Robustness for ReLU Networkscited by 255
- 2021Beta-CROWN: Efficient Bound Propagation with Per-neuron Split Constraints for Complete and Incomplete Neural Network Robustness Verificationcited by 228
- 2020Liquid Time-constant Networkscited by 161
2 more
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026I Can't Believe It's Not Robust: Catastrophic Collapse of Safety Classifiers under Embedding Drift
- 2026Online Shift Detection and Conformal Adaptation for Deployed Safety Classifiers
- 2025Domain-Agnostic Scalable AI Safety Ensuring Framework
- 2024Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems
- 2025Bridging Efficiency and Safety: Formal Verification of Neural Networks with Early Exits
- 2024Unifying Qualitative and Quantitative Safety Verification of DNN-Controlled Systems
- 2026ReSI: Recursive Safety Improvement toward Resistant and Resilient AI
- 2023How Safe Am I Given What I See? Calibrated Prediction of Safety Chances for Image-Controlled Autonomy