mlsanityData to 14 Oct 2026

Protecting Your LLMs with Information Bottleneck

arXiv 2404.13968 ↗First posted 22 Apr 2024cs.CL cs.AI cs.CRcited by 29

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

The advent of large language models (LLMs) has revolutionized the field of natural language processing, yet they might be attacked to produce harmful content. Despite efforts to ethically align LLMs, these are often fragile and can be circumvented by jailbreaking attacks through optimized or manual adversarial prompts. To address this, we introduce the Information Bottleneck Protector (IBProtector), a defense mechanism grounded in the information bottleneck principle, and we modify the objective to avoid trivial solutions. The IBProtector selectively compresses and perturbs prompts, facilitated by a lightweight and trainable extractor, preserving only essential information for the target LLMs to respond with the expected answer. Moreover, we further consider a situation where the gradient is not visible to be compatible with any LLM. Our empirical evaluations show that IBProtector outperforms current defense methods in mitigating jailbreak attempts, without overly affecting response quality or inference speed. Its effectiveness and adaptability across various attack methods and target LLMs underscore the potential of IBProtector as a novel, transferable defense that bolsters the security of LLMs without requiring modifications to the underlying models.

Key facts. Cited by 29 papers in the map (data through 14 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 43 of its 50 references that are in the map. First posted on arXiv 22 Apr 2024.

Builds on

43 of 50 references.

Closest ancestors · 14

Same direction, newest first

  1. 2024EasyJailbreak: A Unified Framework for Jailbreaking Large Language Models18 Mar
  2. 2024Defending Large Language Models against Jailbreak Attacks via Semantic Smoothing25 Feb
  3. 2024Weak-to-Strong Jailbreaking on Large Language Models30 Jan
  4. 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily14 Nov
  5. 2023Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulation6 Nov
  6. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  7. 2023Multilingual Jailbreak Challenges in Large Language Models10 Oct
  8. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks5 Oct
  9. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models3 Oct
  10. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts19 Sep
  11. 2023Defending Against Alignment-Breaking Attacks via Robustly Aligned LLM18 Sep
  12. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher12 Aug
2 more
  1. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  2. 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul

Other references · 29

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  6. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  7. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  8. 2023Mistral 7Bcited by 3,164
  9. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  10. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  11. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,739
  12. 2020Learning to summarize from human feedbackcited by 2,588
17 more
  1. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
  2. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,031
  3. 2016Deep Variational Information Bottleneckcited by 930
  4. 2015Deep Learning and the Information Bottleneck Principlecited by 802
  5. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 651
  6. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 463
  7. 2023Large Language Model Unlearningcited by 302
  8. 2023Pretraining Language Models with Human Preferencescited by 247
  9. 2023LLM Self Defense: By Self Examination, LLMs Know They Are Being Trickedcited by 244
  10. 2023Beyond Memorization: Violating Privacy Via Inference with Large Language Modelscited by 175
  11. 2022Interpretable and Generalizable Graph Learning via Stochastic Attention Mechanismcited by 171
  12. 2024Curiosity-driven Red-teaming for Large Language Modelscited by 88
  13. 2021Improving Question Answering Model Robustness with Synthetic Adversarial Data Generationcited by 79
  14. 2019BottleSum: Unsupervised and Self-supervised Sentence Summarization using the Information Bottleneck Principlecited by 39
  15. 2019Specializing Word Embeddings (for Parsing) by Information Bottleneckcited by 27
  16. 2024Explaining Time Series via Contrastive and Locally Sparse Perturbationscited by 22
  17. 2023Factorized Explainer for Graph Neural Networkscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 8; 2025: 13; 2026: 8.

29 papers.

Sort:

By area:

  1. 2024LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shiftscited by 86
  2. 2024Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Trainingcited by 69
  3. 2025G-Safeguard: A Topology-Guided Security Lens and Treatment on LLM-based Multi-agent Systemscited by 59
  4. 2025Foot-In-The-Door: A Multi-turn Jailbreak for LLMscited by 54
  5. 2024Bag of Tricks: Benchmarking of Jailbreak Attacks on LLMscited by 51
  6. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 46
  7. 2024TimeX++: Learning Time-Series Explanations with Information Bottleneckcited by 33
  8. 2024BlueSuffix: Reinforced Blue Teaming for Vision-Language Models Against Jailbreak Attackscited by 28
  9. 2024Adversarial Tuning: Defending Against Jailbreak Attacks for LLMscited by 26
  10. 2025X-Boundary: Establishing Exact Safety Boundary to Shield LLMs from Multi-Turn Jailbreaks without Compromising Usabilitycited by 20
  11. 2025DETAM: Defending LLMs Against Jailbreak Attacks via Targeted Attention Modificationcited by 10
  12. 2026The Promptware Kill Chain: How Prompt Injections Gradually Evolved Into a Multistep Malware Delivery Mechanismcited by 9
17 more
  1. 2025Evolving Security in LLMs: A Study of Jailbreak Attacks and Defensescited by 8
  2. 2025TeleAI-Safety: A comprehensive LLM jailbreaking benchmark towards attacks, defenses, and evaluationscited by 7
  3. 2025Concept-Level Explainability for Auditing & Steering LLM Responsescited by 6
  4. 2026Sparse Autoencoders are Capable LLM Jailbreak Mitigatorscited by 6
  5. 2025Sysformer: Safeguarding Frozen Large Language Models with Adaptive System Promptscited by 5
  6. 2025Sample-efficient LLM Optimization with Reset Replaycited by 4
  7. 2025MAS-Shield: A Defense Framework for Secure and Efficient LLM MAScited by 3
  8. 2026Diversified Scaling Inference in Time Series Foundation Modelscited by 3
  9. 2024Alignment-Enhanced Decoding:Defending via Token-Level Adaptive Refining of Probability Distributionscited by 2
  10. 2026Mitigating Translationese Bias in Multilingual LLM-as-a-Judge via Disentangled Information Bottleneckcited by 2
  11. 2025LightDefense: A Lightweight Uncertainty-Driven Defense against Jailbreaks via Shifted Token Distributioncited by 1
  12. 2025Strategic Deflection: Defending LLMs from Logit Manipulationcited by 1
  13. 2025Breaking Memorization Barriers in LLM Code Fine-Tuning via Information Bottleneck for Improved Generalizationcited by 1
  14. 2026ChannelGuard: Safe Models Do Not Compose into Safe Multi-Agent Systemscited by 1
  15. 2026LCO: LLM-based Constraint Optimization for Safer Agentic LLMs in Real-world Taskscited by 0
  16. 2026CS-Guard: Benchmarking LLM Guardrails for Code Generation Securitycited by 0
  17. 2026Reactivating Alignment: Defending LLMs from Jailbreaks via Intention-Aware Input-Output Matchingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Attack and defense techniques in large language models: A survey and new perspectives
  2. 2024Breaking Down the Defenses: A Comparative Survey of Attacks on Large Language Models
  3. 2025PandaGuard: Systematic Evaluation of LLM Safety against Jailbreaking Attacks
  4. 2026Open-Weight LLM Fine-Tuning Defenses are Susceptible to Simple Attacks
  5. 2025Paper Summary Attack: Jailbreaking LLMs through LLM Safety Papers
  6. 2024Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editing
  7. 2024Recent Advances in Attack and Defense Approaches of Large Language Models
  8. 2025Adversarial Suffix Filtering: a Defense Pipeline for LLMs