# Normal vs. Adversarial: Salience-based Analysis of Adversarial Samples for Relation Extraction

arXiv: https://arxiv.org/abs/2104.00312 · HTML: https://mlsanity.com/paper/2104.00312/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 64 references that are in the map.
- First posted on arXiv 1 Apr 2021.

## Builds on (most cited first, 27 of 27)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://mlsanity.com/paper/1907.10529/) (2019)
- [Universal Adversarial Triggers for Attacking and Analyzing NLP](https://mlsanity.com/paper/1908.07125/) (2019)
- [Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment](https://mlsanity.com/paper/1907.11932/) (2019)
- [Attention is not not Explanation](https://mlsanity.com/paper/1908.04626/) (2019)
- [TextBugger: Generating Adversarial Text Against Real-world Applications](https://mlsanity.com/paper/1812.05271/) (2018)
- [Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifiers](https://mlsanity.com/paper/1801.04354/) (2018)
- [Analysis Methods in Neural Language Processing: A Survey](https://mlsanity.com/paper/1812.08951/) (2018)
- [Matching the Blanks: Distributional Similarity for Relation Learning](https://mlsanity.com/paper/1906.03158/) (2019)
- [FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation](https://mlsanity.com/paper/1810.10147/) (2018)
- [Explaining Black Box Predictions and Unveiling Data Artifacts through Influence Functions](https://mlsanity.com/paper/2005.06676/) (2020)
- [The elephant in the interpretability room: Why use attention as explanation when we have saliency methods?](https://mlsanity.com/paper/2010.05607/) (2020)
- [Deep Text Classification Can be Fooled](https://mlsanity.com/paper/1704.08006/) (2017)
- [OpenAttack: An Open-source Textual Adversarial Attack Toolkit](https://mlsanity.com/paper/2009.09191/) (2020)
- [Long-tail Relation Extraction via Knowledge Graph Embeddings and Graph Convolution Networks](https://mlsanity.com/paper/1903.01306/) (2019)
- [Meta-Learning with Dynamic-Memory-Based Prototypical Network for Few-Shot Event Detection](https://mlsanity.com/paper/1910.11621/) (2019)
- [Contrastive Triple Extraction with Generative Transformer](https://mlsanity.com/paper/2009.06207/) (2020)
- [Relation Adversarial Network for Low Resource Knowledge Graph Completion](https://mlsanity.com/paper/1911.03091/) (2019)
- [Conceptualized Representation Learning for Chinese Biomedical Text Mining](https://mlsanity.com/paper/2008.10813/) (2020)
- [Bridging Text and Knowledge with Multi-Prototype Embedding for Few-Shot Relational Triple Extraction](https://mlsanity.com/paper/2010.16059/) (2020)
- [Logic-guided Semantic Representation Learning for Zero-Shot Relation Classification](https://mlsanity.com/paper/2010.16068/) (2020)
- [Attention Is (not) All You Need for Commonsense Reasoning](https://mlsanity.com/paper/1905.13497/) (2019)
- [Finding Influential Instances for Distantly Supervised Relation Extraction](https://mlsanity.com/paper/2009.09841/) (2020)
- [Attention-Based Capsule Networks with Dynamic Routing for Relation Extraction](https://mlsanity.com/paper/1812.11321/) (2018)
- [The Devil is the Classifier: Investigating Long Tail Relation Classification with Decoupling Analysis](https://mlsanity.com/paper/2009.07022/) (2020)

## Built on by (most cited first, 4 of 4)

- [Grad-SAM: Explaining Transformers via Gradient Self-Attention Maps](https://mlsanity.com/paper/2204.11073/) (2022)
- [DeepKE: A Deep Learning Based Knowledge Extraction Toolkit for Knowledge Base Population](https://mlsanity.com/paper/2201.03335/) (2022)
- [ROSE: Robust Selective Fine-tuning for Pre-trained Language Models](https://mlsanity.com/paper/2210.09658/) (2022)
- [Disentangled Contrastive Learning for Learning Robust Textual Representations](https://mlsanity.com/paper/2104.04907/) (2021)

## Abstract (arXiv)

Recent neural-based relation extraction approaches, though achieving promising improvement on benchmark datasets, have reported their vulnerability towards adversarial attacks. Thus far, efforts mostly focused on generating adversarial samples or defending adversarial attacks, but little is known about the difference between normal and adversarial samples. In this work, we take the first step to leverage the salience-based method to analyze those adversarial samples. We observe that salience tokens have a direct correlation with adversarial perturbations. We further find the adversarial perturbations are either those tokens not existing in the training set or superficial cues associated with relation labels. To some extent, our approach unveils the characters against adversarial samples. We release an open-source testbed, "DiagnoseAdv" in https://github.com/zjunlp/DiagnoseAdv.