Checking Leakage Witnesses versus Certifying Bounded Non-Leakage
Not on the map. Why?
Abstract (arXiv)
When a language-model audit finds no leak, what is needed to certify non-leakage? We study guarantees over a declared prompt domain under an executable leakage criterion and decoding rule. For general bounded polynomial-time evaluators, a supplied leaking execution is polynomial-time checkable, while leak existence is NP-complete and deterministic certification is coNP-complete. Exact stochastic certification is coNPPP-complete at every fixed rational cutoff in (0,1). Restricting the computation can change these bounds. For example, certification is in coNP when all randomness is a terminal draw from an efficiently computed finite probability table. Attention models admit polynomial-time certification when local dependency windows of logarithmic length precede one global head, given deterministic decoding, fixed vocabulary, exact rational weighted means, a direct binary affine readout and finite-automaton prompt domains. A construction with two global layers instead makes certification coNP-complete over template domains, with one head per layer, polynomial width, logarithmic precision and an inverse-polynomial logit margin. Planted-secret experiments measure what finite audits miss relative to complete references. Among 30 secret--model-state pairs that leak under greedy single-prompt execution on their secret's 4,096-prompt domain, uniformly selecting 256 recorded evaluations per pair misses every leak for an expected 41.06% of these pairs. Batched and single-prompt checks disagree on one complete-domain decision among all 48 fine-tuned pairs, while a same-order repeat reproduces every single-prompt output. These results distinguish computational conditions for certification from the coverage and execution conditions needed to interpret a negative audit.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 12 of its 56 references that are in the map. First posted on arXiv 26 Sep 2026.
Builds on
12 of 56 references.
References · 12
- 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
- 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
- 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 785
- 2017Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networkscited by 744
- 2023The Expressive Power of Transformers with Chain of Thoughtcited by 50
- 2025The Counting Power of Transformerscited by 9
- 2025Parameterized Hardness of Zonotope Containment and Neural Network Verificationcited by 5
- 2026The Head Complexity of Boolean Functions in Single-Layer Attentioncited by 2
- 2019The Complexity of Verifying Loop-Free Programs as Differentially Privatecited by 1
- 2021MAJORITY-3SAT (and Related Problems) in Polynomial Timecited by 1
Built on by
Not cited yet.