# Improving the Robustness of Summarization Systems with Dual Augmentation

arXiv: https://arxiv.org/abs/2306.01090 · HTML: https://mlsanity.com/paper/2306.01090/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 31 references that are in the map.
- First posted on arXiv 1 Jun 2023.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [BERT-ATTACK: Adversarial Attack Against BERT Using BERT](https://mlsanity.com/paper/2004.09984/) (2020)
- [Universal Sentence Encoder](https://mlsanity.com/paper/1803.11175/) (2018)
- [Synthetic and Natural Noise Both Break Neural Machine Translation](https://mlsanity.com/paper/1711.02173/) (2017)
- [QuestEval: Summarization Asks for Fact-based Evaluation](https://mlsanity.com/paper/2103.12693/) (2021)
- [ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training](https://mlsanity.com/paper/2001.04063/) (2020)
- [Professor Forcing: A New Algorithm for Training Recurrent Networks](https://mlsanity.com/paper/1610.09038/) (2016)
- [Neural Text Summarization: A Critical Evaluation](https://mlsanity.com/paper/1908.08960/) (2019)
- [Better Fine-Tuning by Reducing Representational Collapse](https://mlsanity.com/paper/2008.03156/) (2020)
- [BIGPATENT: A Large-Scale Dataset for Abstractive and Coherent Summarization](https://mlsanity.com/paper/1906.03741/) (2019)
- [Robust Neural Machine Translation with Doubly Adversarial Inputs](https://mlsanity.com/paper/1906.02443/) (2019)
- [AdvAug: Robust Adversarial Augmentation for Neural Machine Translation](https://mlsanity.com/paper/2006.11834/) (2020)
- [Towards Robust Neural Machine Translation](https://mlsanity.com/paper/1805.06130/) (2018)
- [Noisy Self-Knowledge Distillation for Text Summarization](https://mlsanity.com/paper/2009.07032/) (2020)
- [Earlier Isn't Always Better: Sub-aspect Analysis on Corpus and System Biases in Summarization](https://mlsanity.com/paper/1908.11723/) (2019)
- [Towards Improving Faithfulness in Abstractive Summarization](https://mlsanity.com/paper/2210.01877/) (2022)
- [Learning towards Selective Data Augmentation for Dialogue Generation](https://mlsanity.com/paper/2303.09719/) (2023)

## Built on by (most cited first, 9 of 9)

- [From Skepticism to Acceptance: Simulating the Attitude Dynamics Toward Fake News](https://mlsanity.com/paper/2403.09498/) (2024)
- [RoTBench: A Multi-Level Benchmark for Evaluating the Robustness of Large Language Models in Tool Learning](https://mlsanity.com/paper/2401.08326/) (2024)
- [The Stepwise Deception: Simulating the Evolution from True News to Fake News with LLM Agents](https://mlsanity.com/paper/2410.19064/) (2024)
- [Leveraging Professional Radiologists' Expertise to Enhance LLMs' Evaluation for Radiology Reports](https://mlsanity.com/paper/2401.16578/) (2024)
- [ScholarChemQA: Unveiling the Power of Language Models in Chemical Research Question Answering](https://mlsanity.com/paper/2407.16931/) (2024)
- [P^3SUM: Preserving Author's Perspective in News Summarization with Diffusion Language Models](https://mlsanity.com/paper/2311.09741/) (2023)
- [Towards Robustness of Text-to-Visualization Translation against Lexical and Phrasal Variability](https://mlsanity.com/paper/2404.07135/) (2024)
- [Attacks against Abstractive Text Summarization Models through Lead Bias and Influence Functions](https://mlsanity.com/paper/2410.20019/) (2024)
- [Diversity Augmentation of Dynamic User Preference Data for Boosting Personalized Text Summarizers](https://mlsanity.com/paper/2510.10082/) (2025)

## Abstract (arXiv)

A robust summarization system should be able to capture the gist of the document, regardless of the specific word choices or noise in the input. In this work, we first explore the summarization models' robustness against perturbations including word-level synonym substitution and noise. To create semantic-consistent substitutes, we propose a SummAttacker, which is an efficient approach to generating adversarial samples based on language models. Experimental results show that state-of-the-art summarization models have a significant decrease in performance on adversarial and noisy test sets. Next, we analyze the vulnerability of the summarization systems and explore improving the robustness by data augmentation. Specifically, the first brittleness factor we found is the poor understanding of infrequent words in the input. Correspondingly, we feed the encoder with more diverse cases created by SummAttacker in the input space. The other factor is in the latent space, where the attacked inputs bring more variations to the hidden states. Hence, we construct adversarial decoder input and devise manifold softmixing operation in hidden space to introduce more diversity. Experimental results on Gigaword and CNN/DM datasets demonstrate that our approach achieves significant improvements over strong baselines and exhibits higher robustness on noisy, attacked, and clean datasets.