mlsanityData to 14 Oct 2026

Revisiting Jailbreaking for Large Language Models: A Representation Engineering Perspective

arXiv 2401.06824 ↗First posted 12 Jan 2024cs.CL cs.AIcited by 48

On the map: LLM safety and security → Jailbreaking

also in 1 more
Abstract (arXiv)

The recent surge in jailbreaking attacks has revealed significant vulnerabilities in Large Language Models (LLMs) when exposed to malicious inputs. While various defense strategies have been proposed to mitigate these threats, there has been limited research into the underlying mechanisms that make LLMs vulnerable to such attacks. In this study, we suggest that the self-safeguarding capability of LLMs is linked to specific activity patterns within their representation space. Although these patterns have little impact on the semantic content of the generated text, they play a crucial role in shaping LLM behavior under jailbreaking attacks. Our findings demonstrate that these patterns can be detected with just a few pairs of contrastive queries. Extensive experimentation shows that the robustness of LLMs against jailbreaking can be manipulated by weakening or strengthening these patterns. Further visual analysis provides additional evidence for our conclusions, providing new insights into the jailbreaking phenomenon. These findings highlight the importance of addressing the potential misuse of open-source LLMs within the community.

Key facts. Cited by 48 papers in the map (data through 14 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 38 of its 56 references that are in the map. First posted on arXiv 12 Jan 2024.

Builds on

38 of 56 references.

Closest ancestors · 11

Same direction, newest first

  1. 2023Token-Level Adversarial Prompt Detection Based on Perplexity Measures and Contextual Information20 Nov
  2. 2023Cognitive Overload: Jailbreaking Large Language Models with Overloaded Logical Thinking16 Nov
  3. 2023Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritization15 Nov
  4. 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily14 Nov
  5. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreaker6 Nov
  6. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  7. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks5 Oct
  8. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher12 Aug
  9. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  10. 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul
  11. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Study23 May

Other references · 27

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  5. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  6. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  7. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  8. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  9. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  10. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
  11. 2022Toy Models of Superpositioncited by 971
  12. 2021Ethical and social risks of harm from Language Modelscited by 966
15 more
  1. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
  2. 2020Aligning AI With Shared Human Valuescited by 723
  3. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 708
  4. 2023The Internal State of an LLM Knows When It's Lyingcited by 659
  5. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 653
  6. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 651
  7. 2023Zephyr: Direct Distillation of LM Alignmentcited by 506
  8. 2023Pretraining Language Models with Human Preferencescited by 247
  9. 2023Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learningcited by 236
  10. 2022On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoningcited by 194
  11. 2023Generative Language Models and Automated Influence Operations: Emerging Threats and Potential Mitigationscited by 158
  12. 2023Inspecting and Editing Knowledge Representations in Language Modelscited by 141
  13. 2023Bergeron: Combating Adversarial Attacks through a Conscience-Based Alignment Frameworkcited by 30
  14. 2023Why do universal adversarial attacks work on large language models?: Geometry might be the answercited by 14
  15. 2023Towards Explainable and Language-Agnostic LLMs: Symbolic Reverse Engineering of Language at Scalecited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 8; 2025: 29; 2026: 11.

48 papers.

Sort:

By area:

  1. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 126
  2. 2025The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independencecited by 115
  3. 2024Coercing LLMs to do and reveal (almost) anythingcited by 89
  4. 2024Large Language Model Safety: A Holistic Surveycited by 59
  5. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 46
  6. 2025AdaSteer: Your Aligned LLM is Inherently an Adaptive Jailbreak Defendercited by 38
  7. 2025Refusal Direction is Universal Across Safety-Aligned Languagescited by 38
  8. 2024Jailbreaking and Mitigation of Vulnerabilities in Large Language Modelscited by 34
  9. 2024"Not Aligned" is Not "Malicious": Being Careful about Hallucinations of Large Language Models' Jailbreakcited by 25
  10. 2025Reasoning as an Adaptive Defense for Safetycited by 23
  11. 2025Understanding and Mitigating Overrefusal in LLMs from an Unveiling Perspective of Safety Decision Boundarycited by 18
  12. 2025Aligned but Blind: Alignment Increases Implicit Bias by Reducing Awareness of Racecited by 15
36 more
  1. 2024AutoJailbreak: Exploring Jailbreak Attacks and Defenses through a Dependency Lenscited by 14
  2. 2025Understanding How Value Neurons Shape the Generation of Specified Values in LLMscited by 13
  3. 2025From LLMs to MLLMs to Agents: A Survey of Emerging Paradigms in Jailbreak Attacks and Defenses within LLM Ecosystemcited by 12
  4. 2025Rethinking Jailbreak Detection of Large Vision Language Models with Representational Contrastive Scoringcited by 11
  5. 2025Jailbreak Attack Initializations as Extractors of Compliance Directionscited by 10
  6. 2025Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safetycited by 10
  7. 2025Universal Jailbreak Suffixes Are Strong Attention Hijackerscited by 10
  8. 2026Analysing the Safety Pitfalls of Steering Vectorscited by 10
  9. 2025Guardians and Offenders: A Survey on Harmful Content Generation and Safety Mitigation of LLMcited by 9
  10. 2025Latent-space adversarial training with post-aware calibration for defending large language models against jailbreak attackscited by 7
  11. 2025Towards the Worst-case Robustness of Large Language Modelscited by 7
  12. 2025IndoSafety: Culturally Grounded Safety for LLMs in Indonesian Languagescited by 7
  13. 2025Jailbreak Transferability Emerges from Shared Representationscited by 6
  14. 2024Feint and Attack: Attention-Based Strategies for Jailbreaking and Protecting LLMscited by 5
  15. 2025CAVGAN: Unifying Jailbreak and Defense of LLMs via Generative Adversarial Attacks on their Internal Representationscited by 5
  16. 2025SafeInt: Shielding Large Language Models from Jailbreak Attacks via Safety-Aware Representation Interventioncited by 4
  17. 2025Vulnerability Mitigation for Safety-Aligned Language Models via Debiasingcited by 3
  18. 2025LatentBreak: Jailbreaking Large Language Models through Latent Space Feedbackcited by 3
  19. 2024Fooling LLM graders into giving better grades through neural activity guided adversarial promptingcited by 2
  20. 2025Beyond External Monitors: Enhancing Transparency of Large Language Models for Easier Monitoringcited by 2
  21. 2026Knowing without Acting: The Disentangled Geometry of Safety Mechanisms in Large Language Modelscited by 2
  22. 2025Information-theoretic Distinctions Between Deception and Confusioncited by 1
  23. 2025N-GLARE: An Non-Generative Latent Representation-Efficient LLM Safety Evaluatorcited by 1
  24. 2025Understanding and Mitigating Over-refusal for Large Language Models via Representation Interventioncited by 1
  25. 2026Depth Charge: Jailbreak Large Language Models from Deep Safety Attention Headscited by 1
  26. 2026GUARD-SLM: Token Activation-Based Defense Against Jailbreak Attacks for Small Language Modelscited by 1
  27. 2026Safety Geometry Collapse in Multimodal LLMs and Adaptive Drift Correctioncited by 1
  28. 2026Refusal Before Decoding: Detecting and Exploiting Refusal Signals in Intermediate LLM Activationscited by 1
  29. 2025Improving LLM Outputs Against Jailbreak Attacks with Expert Model Integrationcited by 0
  30. 2025MEUV: Achieving Fine-Grained Capability Activation in Large Language Models via Mutually Exclusive Unlock Vectorscited by 0
  31. 2025Keep Calm and Avoid Harmful Content: Concept Alignment and Latent Manipulation Towards Safer Answerscited by 0
  32. 2026Exploring Approaches for Detecting Memorization of Recommender System Data in Large Language Modelscited by 0
  33. 2026The Persistent Vulnerability of Aligned AI Systemscited by 0
  34. 2026To Lie or Not to Lie? Investigating The Biased Spread of Global Lies by LLMscited by 0
  35. 2026Dictionary-Aligned Concept Control for Safeguarding Multimodal LLMscited by 0
  36. 2026CodeMimicry: Exploiting Safety Generalization Lag in Large Language Models via Structured Code Completioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Towards Understanding Jailbreak Attacks in LLMs: A Representation Space Analysis
  2. 2025A Domain-Based Taxonomy of Jailbreak Vulnerabilities in Large Language Models
  3. 2024Understanding Jailbreak Success: A Study of Latent Space Dynamics in Large Language Models
  4. 2024A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Models
  5. 2026Breaking the Assumptions: Auditing Input-Side Jailbreak Defenses Against Semantic Attacks
  6. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Survey
  7. 2024Characterizing and Evaluating the Reliability of LLMs against Jailbreak Attacks
  8. 2026Defending Large Language Models Against Jailbreak Attacks via In-Decoding Safety-Awareness Probing