LASH: Adaptive Semantic Hybridization for Black-Box Jailbreaking of Large Language Models
On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks
also in 1 more
Abstract (arXiv)
Jailbreak attacks expose a persistent gap between the intended safety behavior of aligned large language models and their behavior under adversarial prompting. Existing automated methods are increasingly effective but each commits to a single attack family (e.g., one refinement loop, one tree search, one mutation space, or one strategy library) and no single family dominates: the best-performing method shifts across target models and harm categories, suggesting complementary strengths that per-prompt composition could exploit. We introduce LASH (LLM Adaptive Semantic Hybridization), a black-box framework that treats outputs from multiple base attacks as reusable seed prompts and adaptively composes them for each target request. Given a seed pool, LASH searches over seed subsets and softmax-normalized mixture weights; a composition module synthesizes a single candidate prompt, and a derivative-free genetic optimizer updates the weights using black-box target feedback and a two-stage fitness function combining keyword-based refusal detection with LLM-judge scoring. On JailbreakBench, which contains 100 harmful prompts across 10 categories, we evaluate LASH on six common target models. LASH achieves an average attack success rate of 84.5% under keyword-based evaluation and 74.5% under two-stage evaluation, where responses are first filtered for refusals and then scored by an LLM judge for whether they substantively fulfill the original harmful request. LASH outperforms five state-of-the-art baselines on both metrics with only 30 mean target queries. LASH also remains competitive under three defense mechanisms and induces more success-like internal representations. These results suggest that adaptive composition across heterogeneous jailbreak strategies is a promising direction for black-box red-teaming.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 37 of its 42 references that are in the map. First posted on arXiv 20 May 2026.
Builds on
37 of 42 references.
Closest ancestors · 6
Same direction, newest first
- 2024JailPO: A Novel Black-box Jailbreak Framework via Preference Optimization against Aligned LLMs20 Dec
- 2024DrAttack: Prompt Decomposition and Reconstruction Makes Powerful LLM Jailbreakers25 Feb
- 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models3 Oct
- 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Models1 Sep
- 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher12 Aug
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
Other references · 31
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
- 2022Locating and Editing Factual Associations in GPTcited by 2,691
- 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
- 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
- 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
- 2016Understanding intermediate layers using linear classifier probescited by 1,353
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
- 2022Red Teaming Language Models with Language Modelscited by 1,026
19 more
- 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 832
- 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 672
- 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 573
- 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 560
- 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 536
- 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 469
- 2024A StrongREJECT for Empty Jailbreakscited by 385
- 2023DeepInception: Hypnotize Large Language Model to Be Jailbreakercited by 367
- 2023Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discoverycited by 336
- 2023Low-Resource Languages Jailbreak GPT-4cited by 309
- 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easilycited by 259
- 2024ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMscited by 216
- 2024AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMscited by 199
- 2024COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllabilitycited by 175
- 2024AmpleGCG: Learning a Universal and Transferable Generative Model of Adversarial Suffixes for Jailbreaking Both Open and Closed LLMscited by 172
- 2024AdvPrompter: Fast Adaptive Adversarial Prompting for LLMscited by 156
- 2024FlipAttack: Jailbreak LLMs via Flippingcited by 80
- 2024When LLM Meets DRL: Advancing Jailbreaking Efficiency via DRL-guided Searchcited by 47
- 2025LARGO: Latent Adversarial Reflection through Gradient Optimization for Jailbreaking LLMscited by 7
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Adversarial Reframing: A Framework for Targeted Generation in Language Models
- 2025ForgeDAN: An Evolutionary Framework for Jailbreaking Aligned Large Language Models
- 2025HarmNet: A Framework for Adaptive Multi-Turn Jailbreak Attacks on Large Language Models
- 2025Advancing Jailbreak Strategies: A Hybrid Approach to Exploiting LLM Vulnerabilities and Bypassing Modern Defenses
- 2025AutoAdv: Automated Adversarial Prompting for Multi-Turn Jailbreaking of Large Language Models
- 2024AutoBreach: Universal and Adaptive Jailbreaking with Efficient Wordplay-Guided Optimization
- 2025AJF: Adaptive Jailbreak Framework Based on the Comprehension Ability of Black-Box Large Language Models
- 2025Safe2Harm: Semantic Isomorphism Attacks for Jailbreaking Large Language Models