mlsanityData to 7 Oct 2026

RST-LoRA: A Discourse-Aware Low-Rank Adaptation for Long Document Abstractive Summarization

arXiv 2405.00657 ↗First posted 1 May 2024cs.CL cs.AI cs.LGcited by 10

Not on the map. Why?

Abstract (arXiv)

For long document summarization, discourse structure is important to discern the key content of the text and the differences in importance level between sentences. Unfortunately, the integration of rhetorical structure theory (RST) into parameter-efficient fine-tuning strategies for long document summarization remains unexplored. Therefore, this paper introduces RST-LoRA and proposes four RST-aware variants to explicitly incorporate RST into the LoRA model. Our empirical evaluation demonstrates that incorporating the type and uncertainty of rhetorical relations can complementarily enhance the performance of LoRA in summarization tasks. Furthermore, the best-performing variant we introduced outperforms the vanilla LoRA and full-parameter fine-tuning models, as confirmed by multiple automatic and human evaluations, and even surpasses previous state-of-the-art methods.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 57 of its 95 references that are in the map. First posted on arXiv 1 May 2024.

Builds on

57 of 95 references.

References · 57

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  6. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  7. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  8. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  9. 2020Longformer: The Long-Document Transformercited by 3,362
  10. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  11. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  12. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
38 more
  1. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  2. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 896
  3. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 863
  4. 2018A Discourse-Aware Attention Model for Abstractive Summarization of Long Documentscited by 477
  5. 2021SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarizationcited by 402
  6. 2021BookSum: A Collection of Datasets for Long-form Narrative Summarizationcited by 184
  7. 2021UniPELT: A Unified Framework for Parameter-Efficient Language Model Tuningcited by 162
  8. 2019Discourse-Aware Neural Extractive Text Summarizationcited by 104
  9. 2022ATTEMPT: Parameter-Efficient Multi-task Tuning via Attentional Mixtures of Soft Promptscited by 85
  10. 2022Revisiting Parameter-Efficient Tuning: Are We Really There Yet?cited by 76
  11. 2023Conditional Adapters: Parameter-efficient Transfer Learning with Fast Inferencecited by 69
  12. 2023Parameter-Efficient Fine-Tuning Design Spacescited by 67
  13. 2023Parameter-Efficient Fine-Tuning without Introducing New Latencycited by 59
  14. 2022Making Science Simple: Corpora for the Lay Summarisation of Scientific Literaturecited by 54
  15. 2022Investigating Efficiently Extending Transformers for Long Input Summarizationcited by 43
  16. 2022Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infillingcited by 38
  17. 2021Residual Adapters for Parameter-Efficient ASR Adaptation to Atypical and Accented Speechcited by 30
  18. 2023Towards Adaptive Prefix Tuning for Parameter-Efficient Language Model Fine-tuningcited by 29
  19. 2018Discourse-Aware Neural Rewards for Coherent Text Generationcited by 28
  20. 2021DMRST: A Joint Framework for Document-Level Multilingual RST Discourse Segmentation and Parsingcited by 27
  21. 2021Structural Guidance for Transformer Language Modelscited by 25
  22. 2023Neural Architecture Search for Parameter-Efficient Fine-tuning of Large Pre-trained Language Modelscited by 25
  23. 2023RECAP: Retrieval-Enhanced Context-Aware Prefix Encoder for Personalized Dialogue Response Generationcited by 25
  24. 2019Enhancing Machine Translation with Dependency-Aware Self-Attentioncited by 24
  25. 2020Linguistic Profiling of a Neural Language Modelcited by 23
  26. 2022HIBRIDS: Attention with Hierarchical Biases for Structure-aware Long Document Summarizationcited by 22
  27. 2022When a sentence does not introduce a discourse entity, Transformer-based models still sometimes refer to itcited by 22
  28. 2023One Network, Many Masks: Towards More Parameter-Efficient Transfer Learningcited by 18
  29. 2023Incorporating Distributions of Discourse Structure for Long Document Abstractive Summarizationcited by 17
  30. 2017Improved Neural Machine Translation with a Syntax-Aware Encoder and Decodercited by 16
  31. 2022Parameter-Efficient Tuning Makes a Good Classification Headcited by 15
  32. 2023Prefix Propagation: Parameter-Efficient Tuning for Long Sequencescited by 15
  33. 2020Multilingual Neural RST Discourse Parsingcited by 13
  34. 2024SciNews: From Scholarly Complexities to Public Narratives -- A Dataset for Scientific News Report Generationcited by 12
  35. 2019Unsupervised Neural Single-Document Summarization of Reviews via Learning Latent Discourse Structure and its Rankingcited by 11
  36. 2024Infusing Hierarchical Guidance into Prompt Tuning: A Parameter-Efficient Framework for Multi-level Implicit Discourse Relation Recognitioncited by 11
  37. 2022Parameter-Efficient Finetuning for Robust Continual Multilingual Learningcited by 9
  38. 2023Improving Biomedical Abstractive Summarisation with Knowledge Aggregation from Citation Paperscited by 9

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 5; 2026: 4.

10 papers.

  1. 2024A Systematic Survey of Text Summarization: From Statistical Methods to Large Language Modelscited by 52
  2. 2025What Is That Talk About? A Video-to-Text Summarization Dataset for Scientific Presentationscited by 11
  3. 2025Explanatory Summarization with Discourse-Driven Planningcited by 4
  4. 2025F-Adapter: Frequency-Adaptive Parameter-Efficient Fine-Tuning in Scientific Machine Learningcited by 3
  5. 2025LoRAFusion: Efficient LoRA Fine-Tuning for LLMscited by 3
  6. 2025Synthetic Data Augmentation for Cross-domain Implicit Discourse Relation Recognitioncited by 1
  7. 2026Disco-RAG: Discourse-Aware Retrieval-Augmented Generationcited by 1
  8. 2026Automated Summarization of Software Documents: An LLM-based Multi-Agent Approachcited by 1
  9. 2026Summarization is Not Dead Yetcited by 0
  10. 2026Applicability Condition Extraction for Therapeutic Drug-Disease Relationscited by 0