mlsanityData to 7 Oct 2026

Key-Element-Informed sLLM Tuning for Document Summarization

arXiv 2406.04625 ↗First posted 7 Jun 2024cs.CL cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

Remarkable advances in large language models (LLMs) have enabled high-quality text summarization. However, this capability is currently accessible only through LLMs of substantial size or proprietary LLMs with usage fees. In response, smaller-scale LLMs (sLLMs) of easy accessibility and low costs have been extensively studied, yet they often suffer from missing key information and entities, i.e., low relevance, in particular, when input documents are long. We hence propose a key-element-informed instruction tuning for summarization, so-called KEITSum, which identifies key elements in documents and instructs sLLM to generate summaries capturing these key elements. Experimental results on dialogue and news datasets demonstrate that sLLM with KEITSum indeed provides high-quality summarization with higher relevance and less hallucinations, competitive to proprietary LLM.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 35 of its 37 references that are in the map. First posted on arXiv 7 Jun 2024.

Builds on

35 of 37 references.

References · 35

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  7. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  8. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  9. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,100
  10. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 809
  11. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 729
  12. 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 578
23 more
  1. 2019Text Summarization with Pretrained Encoderscited by 553
  2. 2023Benchmarking Large Language Models for News Summarizationcited by 444
  3. 2022News Summarization and Evaluation in the Era of GPT-3cited by 324
  4. 2022Towards a Unified Multi-Dimensional Evaluator for Text Generationcited by 264
  5. 2021QuestEval: Summarization Asks for Fact-based Evaluationcited by 241
  6. 2021DialogSum: A Real-Life Scenario Dialogue Summarization Datasetcited by 181
  7. 2022BRIO: Bringing Order to Abstractive Summarizationcited by 161
  8. 2021Q2: Evaluating Factual Consistency in Knowledge-Grounded Dialogues via Question Generation and Question Answeringcited by 131
  9. 2023Human-like Summarization Evaluation with ChatGPTcited by 126
  10. 2020Evaluating Factuality in Generation with Dependency-level Entailmentcited by 122
  11. 2023Element-aware Summarization with Large Language Models: Expert-aligned Evaluation and Chain-of-Thought Methodcited by 76
  12. 2023Factually Consistent Summarization via Reinforcement Learning with Textual Entailment Feedbackcited by 71
  13. 2019Leveraging BERT for Extractive Text Summarization on Lecturescited by 63
  14. 2021CONFIT: Toward Faithful Dialogue Summarization with Linguistically-Informed Contrastive Fine-tuningcited by 44
  15. 2023Summarization is (Almost) Deadcited by 43
  16. 2023Faithfulness-Aware Decoding Strategies for Abstractive Summarizationcited by 31
  17. 2022Learning to Revise References for Faithful Summarizationcited by 28
  18. 2020Constrained Abstractive Summarization: Preserving Factual Consistency with Constrained Generationcited by 26
  19. 2023Quantifying and Attributing the Hallucination of Large Language Models via Association Analysiscited by 14
  20. 2024Multi-Dimensional Optimization for Text Summarization via Reinforcement Learningcited by 6
  21. 2021Incorporating Question Answering-Based Signals into Abstractive Summarization via Salient Span Selectioncited by 5
  22. 2023Named Entity Inclusion in Abstractive Text Summarizationcited by 3
  23. 2022Towards Understanding Omission in Dialogue Summarizationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 5; 2026: 2.

8 papers.

  1. 2025K-COMP: Retrieval-Augmented Medical Domain Question Answering With Knowledge-Injected Compressorcited by 2
  2. 2025Leveraging What's Overfixed: Post-Correction via LLM Grammatical Error Overcorrectioncited by 2
  3. 2025Adaptive Planning for Multi-Attribute Controllable Summarization with Monte Carlo Tree Searchcited by 2
  4. 2024Exploring Iterative Controllable Summarization with Large Language Modelscited by 1
  5. 2025Towards Prompt Generalization: Grammar-aware Cross-Prompt Automated Essay Scoringcited by 1
  6. 2025Introducing Spotlight: A Novel Approach for Generating Captivating Key Information from Documentscited by 1
  7. 2026Understanding LLM Behavior in Multi-Target Cross-Lingual Summarizationcited by 0
  8. 2026Segment-level Tree Search for Long Meeting Document Summarizationcited by 0