Isolating LLM Alignment from Regex: Zero Coverage and Metric-Dependent Divergence Under Adversarial Mutation
On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks
Abstract (arXiv)
Production LLM applications commonly stack a regex filter in front of model-side alignment; prior work found no measurable coverage gain from adding a live Gemini backend behind an active regex filter. We ask whether that ceiling holds when the corpus is designed to bypass the regex. We introduce L5-no-regex -- identical to L4-real (Gemini-2.5-flash, token-budget cap, rate limit, output scrub) but with the nine-pattern filter disabled -- and evaluate it against N=45 adversarial probes across three sub-corpora (carry-forward, regex-bypass, alignment-isolate), amplified by Gemini paraphrase and PAIR to ~1,555 probe-run pairs over N=5 replications. Under the primary substring classifier, H1 is refuted: L5 block rate is 0,% across all five OWASP LLM Top-10 categories (Deltapp=0 vs. L0, p=1.00; Wilson upper bound <5,%). A secondary LLM-judge metric on PAIR variants shows 56--100,% block rates (p<0.01), revealing alignment does respond to adversarially-framed probes -- but produces refusals too nuanced for substring matching. The sub-corpus differential prediction is not supported (p=1.00). Alignment's contribution is metric-dependent: on natural-language harmful-request probes, it adds zero observed coverage beyond the regex; on adversarially-framed variants, an LLM judge detects refusals the substring classifier misses. The locked corpus, mutation artifacts, and export scripts are released for replication.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 14 of its 28 references that are in the map. First posted on arXiv 12 Jun 2026.
Builds on
14 of 28 references.
Closest ancestors · 3
Same direction, newest first
- 2024SecAlign: Defending Against Prompt Injection with Preference Optimization7 Oct
- 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models3 Oct
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
Other references · 11
- 2020Shortcut Learning in Deep Neural Networkscited by 1,969
- 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
- 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
- 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
- 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 832
- 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 672
- 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 536
- 2024A StrongREJECT for Empty Jailbreakscited by 385
- 2025The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injectionscited by 129
- 2026The Judge Who Never Admits: Hidden Shortcuts in LLM-based Evaluationcited by 3
- 2026Which Defense Closes Which Threat? Attributing OWASP-LLM-Top-10 Coverage and Its Brittleness Under Paraphrasingcited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Efficient Alignment of Large Language Models via Data Sampling
- 2025AdversariaLLM: A Unified and Modular Toolbox for LLM Robustness Research
- 2026Rendering Data Unlearnable by Exploiting LLM Alignment Mechanisms
- 2026Hair-Trigger Alignment: Black-Box Evaluation Cannot Guarantee Post-Update Alignment
- 2025Targeting Alignment: Extracting Safety Classifiers of Aligned LLMs
- 2025LLM Safety Alignment is Divergence Estimation in Disguise
- 2025Adversarial Suffix Filtering: a Defense Pipeline for LLMs
- 2024Aligners: Decoupling LLMs and Alignment