# Adversarial Attack and Defense of Structured Prediction Models

arXiv: https://arxiv.org/abs/2010.01610 · HTML: https://mlsanity.com/paper/2010.01610/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Oct 2020.

## Built on by (most cited first, 20 of 20)

- [Detecting Language Model Attacks with Perplexity](https://mlsanity.com/paper/2308.14132/) (2023)
- [Certified Robustness to Text Adversarial Attacks by Randomized [MASK]](https://mlsanity.com/paper/2105.03743/) (2021)
- [Evaluating the Robustness of Text-to-image Diffusion Models against Real-world Attacks](https://mlsanity.com/paper/2306.13103/) (2023)
- [Code-Mixing on Sesame Street: Dawn of the Adversarial Polyglots](https://mlsanity.com/paper/2103.09593/) (2021)
- [A Survey of Adversarial Defences and Robustness in NLP](https://mlsanity.com/paper/2203.06414/) (2022)
- [Improving the Adversarial Robustness of NLP Models by Information Bottleneck](https://mlsanity.com/paper/2206.05511/) (2022)
- [Collective Robustness Certificates: Exploiting Interdependence in Graph Neural Networks](https://mlsanity.com/paper/2302.02829/) (2023)
- [Token-Modification Adversarial Attacks for Natural Language Processing: A Survey](https://mlsanity.com/paper/2103.00676/) (2021)
- [To Augment or Not to Augment? A Comparative Study on Text Augmentation Techniques for Low-Resource NLP](https://mlsanity.com/paper/2111.09618/) (2021)
- [Constrained Text Generation with Global Guidance -- Case Study on CommonGen](https://mlsanity.com/paper/2103.07170/) (2021)
- [Residue-Based Natural Language Adversarial Attack Detection](https://mlsanity.com/paper/2204.10192/) (2022)
- [Greener yet Powerful: Taming Large Code Generation Models with Quantization](https://mlsanity.com/paper/2303.05378/) (2023)
- [An integrated Auto Encoder-Block Switching defense approach to prevent adversarial attacks](https://mlsanity.com/paper/2203.10930/) (2022)
- [NLP Security and Ethics, in the Wild](https://mlsanity.com/paper/2504.06669/) (2025)
- [Off-The-Shelf Image-to-Image Models Are All You Need To Defeat Image Protection Schemes](https://mlsanity.com/paper/2602.22197/) (2026)
- [SEPP: Similarity Estimation of Predicted Probabilities for Defending and Detecting Adversarial Text](https://mlsanity.com/paper/2110.05748/) (2021)
- [SCAT: Robust Self-supervised Contrastive Learning via Adversarial Training for Text Classification](https://mlsanity.com/paper/2307.01488/) (2023)
- [READ: Improving Relation Extraction from an ADversarial Perspective](https://mlsanity.com/paper/2404.02931/) (2024)
- [IAE: Irony-based Adversarial Examples for Sentiment Analysis Systems](https://mlsanity.com/paper/2411.07850/) (2024)
- [Coordinated Robustness Evaluation Framework for Vision-Language Models](https://mlsanity.com/paper/2506.05429/) (2025)

## Abstract (arXiv)

Building an effective adversarial attacker and elaborating on countermeasures for adversarial attacks for natural language processing (NLP) have attracted a lot of research in recent years. However, most of the existing approaches focus on classification problems. In this paper, we investigate attacks and defenses for structured prediction tasks in NLP. Besides the difficulty of perturbing discrete words and the sentence fluency problem faced by attackers in any NLP tasks, there is a specific challenge to attackers of structured prediction models: the structured output of structured prediction models is sensitive to small perturbations in the input. To address these problems, we propose a novel and unified framework that learns to attack a structured prediction model using a sequence-to-sequence model with feedbacks from multiple reference models of the same structured prediction task. Based on the proposed attack, we further reinforce the victim model with adversarial training, making its prediction more robust and accurate. We evaluate the proposed framework in dependency parsing and part-of-speech tagging. Automatic and human evaluations show that our proposed framework succeeds in both attacking state-of-the-art structured prediction models and boosting them with adversarial training.