mlsanityData to 7 Oct 2026

Information-Theoretic Limits of Safety Verification for Self-Improving Systems

arXiv 2603.28650 ↗First posted 30 Mar 2026cs.LG cs.AI stat.MLcited by 1

Not on the map. Why?

Abstract (arXiv)

Can a safety gate permit unbounded beneficial self-modification while maintaining bounded cumulative risk? We formalize this question through dual conditions -- requiring sum delta_n < infinity (bounded risk) and sum TPR_n = infinity (unbounded utility) -- and establish a theory of their (in)compatibility. Classification impossibility (Theorem 1): For power-law risk schedules delta_n = O(n^{-p}) with p > 1, any classifier-based gate under overlapping safe/unsafe distributions satisfies TPR_n <= C_alpha * delta_n^beta via Holder's inequality, forcing sum TPR_n < infinity. This impossibility is exponent-optimal (Theorem 3). A second independent proof via the NP counting method (Theorem 4) yields a 13% tighter bound without Holder's inequality. Universal finite-horizon ceiling (Theorem 5): For any summable risk schedule, the exact maximum achievable classifier utility is U*(N, B) = N * TPR_NP(B/N), growing as exp(O(sqrt(log N))) -- subpolynomial. At N = 10^6 with budget B = 1.0, a classifier extracts at most U* ~ 87 versus a verifier's ~500,000. Verification escape (Theorem 2): A Lipschitz ball verifier achieves delta = 0 with TPR > 0, escaping the impossibility. Formal Lipschitz bounds for pre-LayerNorm transformers under LoRA enable LLM-scale verification. The separation is strict. We validate on GPT-2 (d_LoRA = 147,456): conditional delta = 0 with TPR = 0.352. Comprehensive empirical validation is in the companion paper [D2].

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 10 of its 39 references that are in the map. First posted on arXiv 30 Mar 2026.

Builds on

10 of 39 references.

References · 10

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2016Concrete Problems in AI Safetycited by 1,582
  3. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  4. 2018Robustness May Be at Odds with Accuracycited by 873
  5. 2018Scalable agent alignment via reward modeling: a research directioncited by 438
  6. 2018Lipschitz regularity of deep neural networks: analysis and efficient estimationcited by 413
  7. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 347
  8. 2019Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networkscited by 290
  9. 2020The Lipschitz Constant of Self-Attentioncited by 171
  10. 2021Lipschitz Normalization for Self-Attention Layers with Application to Graph Neural Networkscited by 38

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Safety Hacking in Constrained Best-of-N Inference-time Scaling
  2. 2026Bounded Reachability & Jailbreak Detection via Contraction-Constrained State Space Models
  3. 2026Falsifiable Release Gates for Self-Improving Systems: Standing Invariants at Scale
  4. 2025Verifiable Safety Q-Filters via Hamilton-Jacobi Reachability and Multiplicative Q-Networks
  5. 2026The Geometry of Refusal: Why Post-Hoc Safety Is Fragile and Pretraining-Time Safety Persists
  6. 2026Lipschitz-Based Robustness Certification Under Floating-Point Execution
  7. 2026Available Guardrails: Certifying Selective Prediction across ML Systems
  8. 2026The Architecture of Errors: From Universal Impossibility to Patch-Local LLM Reliability