# Transforming Wikipedia into Augmented Data for Query-Focused Summarization

arXiv: https://arxiv.org/abs/1911.03324 · HTML: https://mlsanity.com/paper/1911.03324/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Nov 2019.

## Built on by (most cited first, 12 of 12)

- [A Survey of Data Augmentation Approaches for NLP](https://mlsanity.com/paper/2105.03075/) (2021)
- [QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization](https://mlsanity.com/paper/2104.05938/) (2021)
- [A Survey of Data Synthesis Approaches](https://mlsanity.com/paper/2407.03672/) (2024)
- [Text Summarization with Latent Queries](https://mlsanity.com/paper/2106.00104/) (2021)
- [Improved Text Classification via Test-Time Augmentation](https://mlsanity.com/paper/2206.13607/) (2022)
- [WikiDes: A Wikipedia-Based Dataset for Generating Short Descriptions from Paragraphs](https://mlsanity.com/paper/2209.13101/) (2022)
- [On The Persona-based Summarization of Domain-Specific Documents](https://mlsanity.com/paper/2406.03986/) (2024)
- [Surfer100: Generating Surveys From Web Resources, Wikipedia-style](https://mlsanity.com/paper/2112.06377/) (2021)
- [MODABS: Multi-Objective Learning for Dynamic Aspect-Based Summarization](https://mlsanity.com/paper/2406.03479/) (2024)
- [Instructive Dialogue Summarization with Query Aggregations](https://mlsanity.com/paper/2310.10981/) (2023)
- [Aspect-based Meeting Transcript Summarization: A Two-Stage Approach with Weak Supervision on Sentence Classification](https://mlsanity.com/paper/2311.04292/) (2023)
- [UATTA-EB: Uncertainty-Aware Test-Time Augmented Ensemble of BERTs for Classifying Common Mental Illnesses on Social Media Posts](https://mlsanity.com/paper/2304.04539/) (2023)

## Abstract (arXiv)

The limited size of existing query-focused summarization datasets renders training data-driven summarization models challenging. Meanwhile, the manual construction of a query-focused summarization corpus is costly and time-consuming. In this paper, we use Wikipedia to automatically collect a large query-focused summarization dataset (named WIKIREF) of more than 280, 000 examples, which can serve as a means of data augmentation. We also develop a BERT-based query-focused summarization model (Q-BERT) to extract sentences from the documents as summaries. To better adapt a huge model containing millions of parameters to tiny benchmarks, we identify and fine-tune only a sparse subnetwork, which corresponds to a small fraction of the whole model parameters. Experimental results on three DUC benchmarks show that the model pre-trained on WIKIREF has already achieved reasonable performance. After fine-tuning on the specific benchmark datasets, the model with data augmentation outperforms strong comparison systems. Moreover, both our proposed Q-BERT model and subnetwork fine-tuning further improve the model performance. The dataset is publicly available at https://aka.ms/wikiref.