# Don't sweat the small stuff, classify the rest: Sample Shielding to protect text classifiers against adversarial attacks

arXiv: https://arxiv.org/abs/2205.01714 · HTML: https://mlsanity.com/paper/2205.01714/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 22 references that are in the map.
- First posted on arXiv 3 May 2022.

## Builds on (most cited first, 17 of 17)

- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment](https://mlsanity.com/paper/1907.11932/) (2019)
- [BERT-ATTACK: Adversarial Attack Against BERT Using BERT](https://mlsanity.com/paper/2004.09984/) (2020)
- [TextBugger: Generating Adversarial Text Against Real-world Applications](https://mlsanity.com/paper/1812.05271/) (2018)
- [BAE: BERT-based Adversarial Examples for Text Classification](https://mlsanity.com/paper/2004.01970/) (2020)
- [FreeLB: Enhanced Adversarial Training for Natural Language Understanding](https://mlsanity.com/paper/1909.11764/) (2019)
- [Certified Robustness to Adversarial Word Substitutions](https://mlsanity.com/paper/1909.00986/) (2019)
- [Contextualized Perturbation for Textual Adversarial Attack](https://mlsanity.com/paper/2009.07502/) (2020)
- [Text Processing Like Humans Do: Visually Attacking and Shielding NLP Systems](https://mlsanity.com/paper/1903.11508/) (2019)
- [Towards Improving Adversarial Training of NLP Models](https://mlsanity.com/paper/2109.00544/) (2021)
- [Certified Robustness to Text Adversarial Attacks by Randomized [MASK]](https://mlsanity.com/paper/2105.03743/) (2021)
- [Searching for an Effective Defender: Benchmarking Defense against Adversarial Word Substitution](https://mlsanity.com/paper/2108.12777/) (2021)
- [BERT-Defense: A Probabilistic Model Based on BERT to Combat Cognitively Inspired Orthographic Adversarial Attacks](https://mlsanity.com/paper/2106.01452/) (2021)
- [White-to-Black: Efficient Distillation of Black-Box Adversarial Attacks](https://mlsanity.com/paper/1904.02405/) (2019)
- [Defense of Word-level Adversarial Attacks via Random Substitution Encoding](https://mlsanity.com/paper/2005.00446/) (2020)
- [Shielding Google's language toxicity model against adversarial attacks](https://mlsanity.com/paper/1801.01828/) (2018)
- [TREATED:Towards Universal Defense against Textual Adversarial Attacks](https://mlsanity.com/paper/2109.06176/) (2021)

## Built on by (most cited first, 3 of 3)

- [Don't Retrain, Just Rewrite: Countering Adversarial Perturbations by Rewriting Text](https://mlsanity.com/paper/2305.16444/) (2023)
- [CitySpec with Shield: A Secure Intelligent Assistant for Requirement Formalization](https://mlsanity.com/paper/2302.09665/) (2023)
- [RedHerring Attack: Testing the Reliability of Attack Detection](https://mlsanity.com/paper/2509.20691/) (2025)

## Abstract (arXiv)

Deep learning (DL) is being used extensively for text classification. However, researchers have demonstrated the vulnerability of such classifiers to adversarial attacks. Attackers modify the text in a way which misleads the classifier while keeping the original meaning close to intact. State-of-the-art (SOTA) attack algorithms follow the general principle of making minimal changes to the text so as to not jeopardize semantics. Taking advantage of this we propose a novel and intuitive defense strategy called Sample Shielding. It is attacker and classifier agnostic, does not require any reconfiguration of the classifier or external resources and is simple to implement. Essentially, we sample subsets of the input text, classify them and summarize these into a final decision. We shield three popular DL text classifiers with Sample Shielding, test their resilience against four SOTA attackers across three datasets in a realistic threat setting. Even when given the advantage of knowing about our shielding strategy the adversary's attack success rate is <=10% with only one exception and often < 5%. Additionally, Sample Shielding maintains near original accuracy when applied to original texts. Crucially, we show that the `make minimal changes' approach of SOTA attackers leads to critical vulnerabilities that can be defended against with an intuitive sampling strategy.