mlsanityData to 7 Oct 2026

Improving the Adversarial Robustness of NLP Models by Information Bottleneck

arXiv 2206.05511 ↗First posted 11 Jun 2022cs.CLcited by 17

Not on the map. Why?

Abstract (arXiv)

Existing studies have demonstrated that adversarial examples can be directly attributed to the presence of non-robust features, which are highly predictive, but can be easily manipulated by adversaries to fool NLP models. In this study, we explore the feasibility of capturing task-specific robust features, while eliminating the non-robust ones by using the information bottleneck theory. Through extensive experiments, we show that the models trained with our information bottleneck-based method are able to achieve a significant improvement in robust accuracy, exceeding performances of all the previously reported defense methods while suffering almost no performance drop in clean accuracy on SST-2, AGNEWS and IMDB datasets.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 38 of its 64 references that are in the map. First posted on arXiv 11 Jun 2022.

Builds on

38 of 64 references.

References · 38

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  5. 2013Intriguing properties of neural networkscited by 5,114
  6. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  7. 2016Adversarial examples in the physical worldcited by 1,762
  8. 2014How transferable are features in deep neural networks?cited by 1,216
  9. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
  10. 2016Deep Variational Information Bottleneckcited by 915
  11. 2018Robustness May Be at Odds with Accuracycited by 873
  12. 2015Deep Learning and the Information Bottleneck Principlecited by 793
26 more
  1. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
  2. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 663
  3. 2019Adversarial Training for Free!cited by 638
  4. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 568
  5. 2020BERT-ATTACK: Adversarial Attack Against BERT Using BERTcited by 488
  6. 2018Generating Natural Language Adversarial Examplescited by 422
  7. 2019PAWS: Paraphrase Adversaries from Word Scramblingcited by 395
  8. 2018TextBugger: Generating Adversarial Text Against Real-world Applicationscited by 376
  9. 2018Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifierscited by 341
  10. 2018Adversarial Example Generation with Syntactically Controlled Paraphrase Networkscited by 282
  11. 2016Adversarial Training Methods for Semi-Supervised Text Classificationcited by 280
  12. 2019FreeLB: Enhanced Adversarial Training for Natural Language Understandingcited by 261
  13. 2019SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimizationcited by 253
  14. 2019Certified Robustness to Adversarial Word Substitutionscited by 164
  15. 2016Information Dropout: Learning Optimal Representations Through Noisy Computationcited by 131
  16. 2018Are adversarial examples inevitable?cited by 113
  17. 2020InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspectivecited by 92
  18. 2019Achieving Verified Robustness to Symbol Substitutions via Interval Bound Propagationcited by 87
  19. 2020An Information Bottleneck Approach for Controlling Conciseness in Rationale Extractioncited by 69
  20. 2018Interpretable Adversarial Perturbation in Input Embedding Space for Textcited by 67
  21. 2019BottleSum: Unsupervised and Self-supervised Sentence Summarization using the Information Bottleneck Principlecited by 39
  22. 2020Defense against Adversarial Attacks in NLP via Dirichlet Neighborhood Ensemblecited by 26
  23. 2020Adversarial Attack and Defense of Structured Prediction Modelscited by 20
  24. 2018AdvEntuRe: Adversarial Training for Textual Entailment with Knowledge-Guided Examplescited by 14
  25. 2018Word Embedding Perturbation for Sentence Classificationcited by 11
  26. 2020Defense of Word-level Adversarial Attacks via Random Substitution Encodingcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 7; 2024: 3; 2025: 4; 2026: 2.

17 papers.

Sort:

By area:

  1. 2024Towards Tracing Trustworthiness Dynamics: Revisiting Pre-training Period of Large Language Modelscited by 18
  2. 2025Mitigating Visual Knowledge Forgetting in MLLM Instruction-tuning via Modality-decoupled Gradient Descentcited by 13
  3. 2023Improving the Robustness of Transformer-based Large Language Models with Dynamic Attentioncited by 10
  4. 2023Enhancing Multimodal Entity and Relation Extraction with Variational Information Bottleneckcited by 9
  5. 2024A Curious Case of Searching for the Correlation between Training Data and Adversarial Robustness of Transformer Textual Modelscited by 5
  6. 2023Span-based Named Entity Recognition by Generating and Compressing Informationcited by 3
  7. 2025VIBE: Annotation-Free Video-to-Text Information Bottleneck Evaluation for TL;DRcited by 3
  8. 2023Unsupervised Sentence Representation Learning with Frequency-induced Adversarial Tuning and Incomplete Sentence Filteringcited by 2
  9. 2023Enhancing Phrase Representation by Information Bottleneck Guided Text Diffusion Process for Keyphrase Extractioncited by 2
  10. 2023Discourse over Discourse: The Need for an Expanded Pragmatic Focus in Conversational AIcited by 1
  11. 2023Fooling the Textual Fooler via Randomizing Latent Representationscited by 1
  12. 2024Exploring Adversarial Robustness in Classification tasks using DNA Language Modelscited by 1
5 more
  1. 2026Off-The-Shelf Image-to-Image Models Are All You Need To Defeat Image Protection Schemescited by 1
  2. 2022Using Large Pre-Trained Language Model to Assist FDA in Premarket Medical Devicecited by 0
  3. 2025Data Security in Large Language Models: Risks, Defense, and Directionscited by 0
  4. 2025Adversarially Robust Detection of Harmful Online Content: A Computational Design Science Approachcited by 0
  5. 2026Evaluation of Adversarial Robustness in Arabic Language Modelscited by 0