# Natural Language Adversarial Defense through Synonym Encoding

arXiv: https://arxiv.org/abs/1909.06723 · HTML: https://mlsanity.com/paper/1909.06723/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Sep 2019.

## Built on by (most cited first, 30 of 40)

- [SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks](https://mlsanity.com/paper/2310.03684/) (2023)
- [Why Should Adversarial Perturbations be Imperceptible? Rethink the Research Paradigm in Adversarial NLP](https://mlsanity.com/paper/2210.10683/) (2022)
- [Defending Against Unforeseen Failure Modes with Latent Adversarial Training](https://mlsanity.com/paper/2403.05030/) (2024)
- [Structure Invariant Transformation for better Adversarial Transferability](https://mlsanity.com/paper/2309.14700/) (2023)
- [Triangle Attack: A Query-efficient Decision-based Adversarial Attack](https://mlsanity.com/paper/2112.06569/) (2021)
- [Text-CRS: A Generalized Certified Robustness Framework against Textual Adversarial Attacks](https://mlsanity.com/paper/2307.16630/) (2023)
- [The Trickle-down Impact of Reward (In-)consistency on RLHF](https://mlsanity.com/paper/2309.16155/) (2023)
- [On the Duality Between Sharpness-Aware Minimization and Adversarial Training](https://mlsanity.com/paper/2402.15152/) (2024)
- [A Survey of Adversarial Defences and Robustness in NLP](https://mlsanity.com/paper/2203.06414/) (2022)
- [Token-Modification Adversarial Attacks for Natural Language Processing: A Survey](https://mlsanity.com/paper/2103.00676/) (2021)
- [Robust Textual Embedding against Word-level Adversarial Attacks](https://mlsanity.com/paper/2202.13817/) (2022)
- [Adversarial Training for Improving Model Robustness? Look at Both Prediction and Interpretation](https://mlsanity.com/paper/2203.12709/) (2022)
- [TextHacker: Learning based Hybrid Local Search Algorithm for Text Hard-label Adversarial Attack](https://mlsanity.com/paper/2201.08193/) (2022)
- [Spiking Convolutional Neural Networks for Text Classification](https://mlsanity.com/paper/2406.19230/) (2024)
- [TextDefense: Adversarial Text Detection based on Word Importance Entropy](https://mlsanity.com/paper/2302.05892/) (2023)
- [Model-tuning Via Prompts Makes NLP Models Adversarially Robust](https://mlsanity.com/paper/2303.07320/) (2023)
- [Detecting Textual Adversarial Examples through Randomized Substitution and Vote](https://mlsanity.com/paper/2109.05698/) (2021)
- [TextShield: Beyond Successfully Detecting Adversarial Sentences in Text Classification](https://mlsanity.com/paper/2302.02023/) (2023)
- [How Vulnerable Are Automatic Fake News Detection Methods to Adversarial Attacks?](https://mlsanity.com/paper/2107.07970/) (2021)
- [Fast Adversarial Training against Textual Adversarial Attacks](https://mlsanity.com/paper/2401.12461/) (2024)
- [Masked Language Model Based Textual Adversarial Example Detection](https://mlsanity.com/paper/2304.08767/) (2023)
- [Using Random Perturbations to Mitigate Adversarial Attacks on Sentiment Analysis Models](https://mlsanity.com/paper/2202.05758/) (2022)
- [Backdoor Learning for NLP: Recent Advances, Challenges, and Future Research Directions](https://mlsanity.com/paper/2302.06801/) (2023)
- [Exploring Gradient-Guided Masked Language Model to Detect Textual Adversarial Attacks](https://mlsanity.com/paper/2504.08798/) (2025)
- [The Best Defense is Attack: Repairing Semantics in Textual Adversarial Examples](https://mlsanity.com/paper/2305.04067/) (2023)
- [From Adversarial Arms Race to Model-centric Evaluation: Motivating a Unified Automatic Robustness Evaluation Framework](https://mlsanity.com/paper/2305.18503/) (2023)
- [SemRoDe: Macro Adversarial Training to Learn Representations That are Robust to Word-Level Attacks](https://mlsanity.com/paper/2403.18423/) (2024)
- [Guided Perturbation Sensitivity (GPS): Detecting Adversarial Text via Embedding Stability and Word Importance](https://mlsanity.com/paper/2508.11667/) (2025)
- [Alert-ME: An Explainability-Driven Defense Against Adversarial Examples in Transformer-Based Text Classification](https://mlsanity.com/paper/2307.01225/) (2023)
- [Bridging Robustness and Generalization Against Word Substitution Attacks in NLP via the Growth Bound Matrix Approach](https://mlsanity.com/paper/2507.10330/) (2025)

## Abstract (arXiv)

In the area of natural language processing, deep learning models are recently known to be vulnerable to various types of adversarial perturbations, but relatively few works are done on the defense side. Especially, there exists few effective defense method against the successful synonym substitution based attacks that preserve the syntactic structure and semantic information of the original text while fooling the deep learning models. We contribute in this direction and propose a novel adversarial defense method called Synonym Encoding Method (SEM). Specifically, SEM inserts an encoder before the input layer of the target model to map each cluster of synonyms to a unique encoding and trains the model to eliminate possible adversarial perturbations without modifying the network architecture or adding extra data. Extensive experiments demonstrate that SEM can effectively defend the current synonym substitution based attacks and block the transferability of adversarial examples. SEM is also easy and efficient to scale to large models and big datasets.