mlsanityData to 7 Oct 2026

QueryAttack: Jailbreaking Aligned Large Language Models Using Structured Non-natural Query Language

arXiv 2502.09723 ↗First posted 13 Feb 2025cs.CR cs.AI cs.CLcited by 16

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

Recent advances in large language models (LLMs) have demonstrated remarkable potential in the field of natural language processing. Unfortunately, LLMs face significant security and ethical risks. Although techniques such as safety alignment are developed for defense, prior researches reveal the possibility of bypassing such defenses through well-designed jailbreak attacks. In this paper, we propose QueryAttack, a novel framework to examine the generalizability of safety alignment. By treating LLMs as knowledge databases, we translate malicious queries in natural language into structured non-natural query language to bypass the safety alignment mechanisms of LLMs. We conduct extensive experiments on mainstream LLMs, and the results show that QueryAttack not only can achieve high attack success rates (ASRs), but also can jailbreak various defense methods. Furthermore, we tailor a defense method against QueryAttack, which can reduce ASR by up to 64% on GPT-4-1106. Our code is available at https://github.com/horizonsinzqs/QueryAttack.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 31 of its 43 references that are in the map. First posted on arXiv 13 Feb 2025.

Builds on

31 of 43 references.

Closest ancestors · 5

Same direction, newest first

  1. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations10 Oct
  2. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!5 Oct
  3. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Models1 Sep
  4. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher12 Aug
  5. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul

Other references · 26

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  3. 2017Deep reinforcement learning from human preferencescited by 3,846
  4. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  5. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  6. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  7. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  8. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 434
  9. 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 358
  10. 2023Multi-step Jailbreaking Privacy Attacks on ChatGPTcited by 320
  11. 2023Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervisioncited by 318
  12. 2024SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decodingcited by 233
14 more
  1. 2023LLM Self Defense: By Self Examination, LLMs Know They Are Being Trickedcited by 226
  2. 2023Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulationcited by 202
  3. 2024ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMscited by 197
  4. 2023Automatically Auditing Large Language Models via Discrete Optimizationcited by 187
  5. 2023Cross-lingual Prompting: Improving Zero-shot Chain-of-Thought Reasoning across Languagescited by 96
  6. 2022Interpreting Language Models with Contrastive Explanationscited by 91
  7. 2023FLIRT: Feedback Loop In-context Red Teamingcited by 81
  8. 2024PsySafe: A Comprehensive Framework for Psychological-based Attack, Defense, and Evaluation of Multi-agent System Safetycited by 79
  9. 2024CodeAttack: Revealing Safety Generalization Challenges of Large Language Models via Code Completioncited by 70
  10. 2023BadCLIP: Trigger-Aware Prompt Learning for Backdoor Attacks on CLIPcited by 63
  11. 2024Identifying Semantic Induction Heads to Understand In-Context Learningcited by 46
  12. 2024The Dark Side of Function Calling: Pathways to Jailbreaking Large Language Modelscited by 20
  13. 2024Embedding Self-Correction as an Inherent Ability in Large Language Models for Enhanced Mathematical Reasoningcited by 13
  14. 2025Dagger Behind Smile: Fool LLMs with a Happy Ending Storycited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 9; 2026: 7.

16 papers.

Sort:

By area:

  1. 2025Foot-In-The-Door: A Multi-turn Jailbreak for LLMscited by 50
  2. 2025MixAT: Combining Continuous and Discrete Adversarial Training for LLMscited by 10
  3. 2025Anyone Can Jailbreak: Prompt-Based Attacks on LLMs and T2Iscited by 6
  4. 2025Benchmarking MLLM-based Web Understanding: Reasoning, Robustness and Safetycited by 5
  5. 2025Imperceptible Jailbreaking against Large Language Modelscited by 3
  6. 2025When Smiley Turns Hostile: Interpreting How Emojis Trigger LLMs' Toxicitycited by 2
  7. 2025Jailbreaking LLMs via Semantically Relevant Nested Scenarios with Targeted Toxic Knowledgecited by 2
  8. 2026Jailbreak Foundry: From Papers to Runnable Attacks for Reproducible Benchmarkingcited by 2
  9. 2026Type-Checked Compliance: Deterministic Guardrails for Agentic Financial Systems Using Lean 4 Theorem Provingcited by 2
  10. 2026SoK: Robustness in Large Language Models against Jailbreak Attackscited by 2
  11. 2026MRMAD: A Multi-Round Multi-Audio Benchmark for Evaluating Acoustic Degradation Perception in Large Audio-Language Modelscited by 2
  12. 2025Beyond Surface-Level Detection: Towards Cognitive-Driven Defense Against Jailbreak Attacks via Meta-Operations Reasoningcited by 1
4 more
  1. 2026Depth Charge: Jailbreak Large Language Models from Deep Safety Attention Headscited by 1
  2. 2025Semantic-aware Graph-guided Behavior Sequences Generation with Large Language Models for Smart Homescited by 0
  3. 2026The Shadow Self: Intrinsic Value Misalignment in Large Language Model Agentscited by 0
  4. 2026Model-Agnostic Lifelong LLM Safety via Externalized Attack-Defense Co-Evolutioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025StructTransform: A Scalable Attack Surface for Safety-Aligned Large Language Models
  2. 2024Jailbreaking Large Language Models Through Alignment Vulnerabilities in Out-of-Distribution Settings
  3. 2025When Style Breaks Safety: Defending LLMs Against Superficial Style Alignment
  4. 2025Beyond Jailbreaks: Revealing Stealthier and Broader LLM Security Risks Stemming from Alignment Failures
  5. 2024Probing the Safety Response Boundary of Large Language Models via Unsafe Decoding Path Generation
  6. 2024WordGame: Efficient & Effective LLM Jailbreak via Simultaneous Obfuscation in Query and Response
  7. 2024Safety Layers in Aligned Large Language Models: The Key to LLM Security
  8. 2025Logic Jailbreak: Efficiently Unlocking LLM Safety Restrictions Through Formal Logical Expression