mlsanityData to 7 Oct 2026

A Guide To Effectively Leveraging LLMs for Low-Resource Text Summarization: Data Augmentation and Semi-supervised Approaches

arXiv 2407.07341 ↗First posted 10 Jul 2024cs.CL cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

Existing approaches for low-resource text summarization primarily employ large language models (LLMs) like GPT-3 or GPT-4 at inference time to generate summaries directly; however, such approaches often suffer from inconsistent LLM outputs and are difficult to adapt to domain-specific data in low-resource scenarios. In this work, we propose two novel methods to effectively utilize LLMs for low-resource text summarization: 1) MixSumm, an LLM-based data augmentation regime that synthesizes high-quality documents (short and long) for few-shot text summarization, and 2) PPSL, a prompt-based pseudolabeling strategy for sample-efficient semi-supervised text summarization. Specifically, MixSumm leverages the open-source LLaMA-3-70b-Instruct model to generate new documents by mixing topical information derived from a small seed set, and PPSL leverages the LLaMA-3-70b-Instruct model to generate high-quality pseudo-labels in a semi-supervised learning setup. We evaluate our methods on the TweetSumm, WikiHow, and ArXiv/PubMed datasets and use L-Eval, a LLaMA-3-based evaluation metric, and ROUGE scores to measure the quality of generated summaries. Our experiments on extractive and abstractive summarization show that MixSumm and PPSL achieve competitive ROUGE scores as a fully supervised method with 5% of the labeled data.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 38 of its 57 references that are in the map. First posted on arXiv 10 Jul 2024.

Builds on

38 of 57 references.

References · 38

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  5. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  6. 2020Longformer: The Long-Document Transformercited by 3,362
  7. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  8. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 809
  9. 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 800
  10. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 721
  11. 2021BARTScore: Evaluating Generated Text as Text Generationcited by 719
  12. 2015Cyclical Learning Rates for Training Neural Networkscited by 627
26 more
  1. 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 578
  2. 2019Text Summarization with Pretrained Encoderscited by 553
  3. 2018A Discourse-Aware Attention Model for Abstractive Summarization of Long Documentscited by 477
  4. 2023Benchmarking Large Language Models for News Summarizationcited by 444
  5. 2021A Survey of Data Augmentation Approaches for NLPcited by 408
  6. 2022News Summarization and Evaluation in the Era of GPT-3cited by 324
  7. 2022Large Language Models Are Reasoning Teacherscited by 318
  8. 2016Adversarial Training Methods for Semi-Supervised Text Classificationcited by 280
  9. 2018WikiHow: A Large Scale Text Summarization Datasetcited by 203
  10. 2020Extractive Summarization as Text Matchingcited by 189
  11. 2021GPT3Mix: Leveraging Large-scale Language Models for Text Augmentationcited by 185
  12. 2020GSum: A General Framework for Guided Neural Abstractive Summarizationcited by 129
  13. 2023AugGPT: Leveraging ChatGPT for Text Data Augmentationcited by 128
  14. 2023Automatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled Datacited by 119
  15. 2021An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Modelscited by 96
  16. 2023On Learning to Summarize with Large Language Models as Referencescited by 80
  17. 2021Medically Aware GPT-3 as a Data Generator for Medical Dialogue Summarizationcited by 76
  18. 2023Benchmarking Generation and Evaluation Capabilities of Large Language Models for Instruction Controllable Summarizationcited by 69
  19. 2020Semi-supervised Medical Image Classification with Relation-driven Self-ensembling Modelcited by 59
  20. 2020Improving Zero and Few-Shot Abstractive Summarization with Intermediate Fine-tuning and Data Augmentationcited by 53
  21. 2022Data Augmentation for Intent Classification with Off-the-shelf Large Language Modelscited by 49
  22. 2023ChatGPT vs Human-authored Text: Insights into Controllable Text Summarization and Sentence Style Transfercited by 49
  23. 2021TWEETSUMM -- A Dialog Summarization Dataset for Customer Servicecited by 34
  24. 2023PromptMix: A Class Boundary Augmentation Method for Large Language Model Distillationcited by 25
  25. 2017Scientific Article Summarization Using Citation-Context and Article's Discourse Structurecited by 17
  26. 2023LLM aided semi-supervision for Extractive Dialog Summarizationcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 3; 2026: 2.

6 papers.

  1. 2024Use Random Selection for Now: Investigation of Few-Shot Selection Strategies in LLM-based Text Augmentation for Classificationcited by 7
  2. 2025A Unifying Scheme for Extractive Content Selection Taskscited by 1
  3. 2025Paired by the Teacher: Turning Unpaired Data into High-Fidelity Pairs for Low-Resource Text Generationcited by 1
  4. 2025Diversity Augmentation of Dynamic User Preference Data for Boosting Personalized Text Summarizerscited by 0
  5. 2026LLMs Should Incorporate Explicit Mechanisms for Human Empathycited by 0
  6. 2026A Stepwise Questioning Expert-Editor Multi-Agent Framework for Long-Document Summarizationcited by 0