# Improving Robustness by Augmenting Training Sentences with Predicate-Argument Structures

arXiv: https://arxiv.org/abs/2010.12510 · HTML: https://mlsanity.com/paper/2010.12510/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Oct 2020.

## Built on by (most cited first, 10 of 10)

- [A Survey of Data Augmentation Approaches for NLP](https://mlsanity.com/paper/2105.03075/) (2021)
- [Generalization in NLI: Ways (Not) To Go Beyond Simple Heuristics](https://mlsanity.com/paper/2110.01518/) (2021)
- [End-to-End Self-Debiasing Framework for Robust NLU Training](https://mlsanity.com/paper/2109.02071/) (2021)
- [Avoiding Inference Heuristics in Few-shot Prompt-based Finetuning](https://mlsanity.com/paper/2109.04144/) (2021)
- [Context-aware Adversarial Training for Name Regularity Bias in Named Entity Recognition](https://mlsanity.com/paper/2107.11610/) (2021)
- [Lexical Generalization Improves with Larger Models and Longer Training](https://mlsanity.com/paper/2210.12673/) (2022)
- [PECO: Examining Single Sentence Label Leakage in Natural Language Inference Datasets through Progressive Evaluation of Cluster Outliers](https://mlsanity.com/paper/2112.09237/) (2021)
- [Does Entity Abstraction Help Generative Transformers Reason?](https://mlsanity.com/paper/2201.01787/) (2022)
- [NATURE: Natural Auxiliary Text Utterances for Realistic Spoken Language Evaluation](https://mlsanity.com/paper/2111.05196/) (2021)
- [Improving Classifier Robustness through Active Generation of Pairwise Counterfactuals](https://mlsanity.com/paper/2305.13535/) (2023)

## Abstract (arXiv)

Existing NLP datasets contain various biases, and models tend to quickly learn those biases, which in turn limits their robustness. Existing approaches to improve robustness against dataset biases mostly focus on changing the training objective so that models learn less from biased examples. Besides, they mostly focus on addressing a specific bias, and while they improve the performance on adversarial evaluation sets of the targeted bias, they may bias the model in other ways, and therefore, hurt the overall robustness. In this paper, we propose to augment the input sentences in the training data with their corresponding predicate-argument structures, which provide a higher-level abstraction over different realizations of the same meaning and help the model to recognize important parts of sentences. We show that without targeting a specific bias, our sentence augmentation improves the robustness of transformer models against multiple biases. In addition, we show that models can still be vulnerable to the lexical overlap bias, even when the training data does not contain this bias, and that the sentence augmentation also improves the robustness in this scenario. We will release our adversarial datasets to evaluate bias in such a scenario as well as our augmentation scripts at https://github.com/UKPLab/data-augmentation-for-robustness.