mlsanityData to 7 Oct 2026

Logicbreaks: A Framework for Understanding Subversion of Rule-based Inference

arXiv 2407.00075 ↗First posted 21 Jun 2024cs.AI cs.CL cs.CRcited by 3

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

We study how to subvert large language models (LLMs) from following prompt-specified rules. We first formalize rule-following as inference in propositional Horn logic, a mathematical system in which rules have the form "if P and Q, then R" for some propositions P, Q, and R. Next, we prove that although small transformers can faithfully follow such rules, maliciously crafted prompts can still mislead both theoretical constructions and models learned from data. Furthermore, we demonstrate that popular attack algorithms on LLMs find adversarial prompts and induce attention patterns that align with our theory. Our novel logic-based framework provides a foundation for studying LLMs in rule-based settings, enabling a formal analysis of tasks like logical reasoning and jailbreak attacks.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 44 of its 57 references that are in the map. First posted on arXiv 21 Jun 2024.

Builds on

44 of 57 references.

Closest ancestors · 9

Same direction, newest first

  1. 2024Don't Say No: Jailbreaking LLM by Suppressing Refusal25 Apr
  2. 2024LLMs Can Defend Themselves Against Jailbreaking in a Practical Manner: A Vision Paper24 Feb
  3. 2024JailbreakRadar: Comprehensive Assessment of Jailbreak Attacks Against LLMs8 Feb
  4. 2023Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritization15 Nov
  5. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  6. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation10 Oct
  7. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks5 Oct
  8. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  9. 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul

Other references · 35

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  7. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  8. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  9. 2013Intriguing properties of neural networkscited by 5,114
  10. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  11. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  12. 2023Mistral 7Bcited by 3,065
23 more
  1. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  2. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  3. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 684
  4. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 394
  5. 2023Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspectivecited by 293
  6. 2019Theoretical Limitations of Self-Attention in Neural Sequence Modelscited by 286
  7. 2023Faithful Chain-of-Thought Reasoningcited by 285
  8. 2022Transformers Learn Shortcuts to Automatacited by 260
  9. 2022The Parallelism Tradeoff: Limitations of Log-Precision Transformerscited by 191
  10. 2023Automatically Auditing Large Language Models via Discrete Optimizationcited by 187
  11. 2023Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Futurecited by 184
  12. 2023Deductive Verification of Chain-of-Thought Reasoningcited by 122
  13. 2023Automatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled Datacited by 119
  14. 2022Overcoming a Theoretical Limitation of Self-Attentioncited by 110
  15. 2020Adversarial Training for Large Neural Language Modelscited by 109
  16. 2022On the Paradox of Learning to Reason from Datacited by 93
  17. 2022Formal Language Recognition by Hard Attention Transformers: Perspectives from Circuit Complexitycited by 92
  18. 2023Reprompting: Automated Chain-of-Thought Prompt Inference Through Gibbs Samplingcited by 46
  19. 2023Boosting Logical Reasoning in Large Language Models through a New Framework: The Graph of Thoughtcited by 41
  20. 2023On the Expressivity Role of LayerNorm in Transformers' Attentioncited by 32
  21. 2024The Ethics of Interaction: Mitigating Security Threats in LLMscited by 30
  22. 2023Average-Hard Attention Transformers are Constant-Depth Uniform Threshold Circuitscited by 26
  23. 2024Evaluation Ethics of LLMs in Legal Domaincited by 8

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 1.

3 papers.

  1. 2024A Implies B: Circuit Analysis in LLMs for Propositional Logical Reasoningcited by 15
  2. 2024Model-Editing-Based Jailbreak against Safety-aligned Large Language Modelscited by 4
  3. 2025Adaptively profiling models with task elicitationcited by 4

Similar, not cited

Close in wording, no citation either way

  1. 2024Reasoning as a Weapon: Adaptive Dual-Path Jailbreak Attack on Large Language Models
  2. 2025Patterns Over Principles: The Fragility of Inductive Reasoning in LLMs under Noisy Observations
  3. 2025Turning Logic Against Itself : Probing Model Defenses Through Contrastive Questions
  4. 2026STAR: Detecting Inference-time Backdoors in LLM Reasoning via State-Transition Amplification Ratio
  5. 2025Formalization Driven LLM Prompt Jailbreaking via Reinforcement Learning
  6. 2025POT: Inducing Overthinking in LLMs via Black-Box Iterative Optimization
  7. 2025Adversarial Reasoning at Jailbreaking Time
  8. 2025Chain-of-Code Collapse: Reasoning Failures in LLMs via Adversarial Prompting in Code Generation