# Attention Optimization for Abstractive Document Summarization

arXiv: https://arxiv.org/abs/1910.11491 · HTML: https://mlsanity.com/paper/1910.11491/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Oct 2019.

## Built on by (most cited first, 4 of 4)

- [DreamStory: Open-Domain Story Visualization by LLM-Guided Multi-Subject Consistent Diffusion](https://mlsanity.com/paper/2407.12899/) (2024)
- [The Summary Loop: Learning to Write Abstractive Summaries Without Examples](https://mlsanity.com/paper/2105.05361/) (2021)
- [Summarising Historical Text in Modern Languages](https://mlsanity.com/paper/2101.10759/) (2021)
- [Reinforcing Semantic-Symmetry for Document Summarization](https://mlsanity.com/paper/2112.07583/) (2021)

## Abstract (arXiv)

Attention plays a key role in the improvement of sequence-to-sequence-based document summarization models. To obtain a powerful attention helping with reproducing the most salient information and avoiding repetitions, we augment the vanilla attention model from both local and global aspects. We propose an attention refinement unit paired with local variance loss to impose supervision on the attention model at each decoding step, and a global variance loss to optimize the attention distributions of all decoding steps from the global perspective. The performances on the CNN/Daily Mail dataset verify the effectiveness of our methods.