mlsanityData to 14 Oct 2026

Learning to Summarize from LLM-generated Feedback

arXiv 2410.13116 ↗First posted 17 Oct 2024cs.CL cs.AIcited by 24

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

Developing effective text summarizers remains a challenge due to issues like hallucinations, key information omissions, and verbosity in LLM-generated summaries. This work explores using LLM-generated feedback to improve summary quality by aligning the summaries with human preferences for faithfulness, completeness, and conciseness. We introduce FeedSum, a large-scale dataset containing multi-dimensional LLM feedback on summaries of varying quality across diverse domains. Our experiments show how feedback quality, dimensionality, and granularity influence preference learning, revealing that high-quality, multi-dimensional, fine-grained feedback significantly improves summary generation. We also compare two methods for using this feedback: supervised fine-tuning and direct preference optimization. Finally, we introduce SummLlama3-8b, a model that outperforms the nearly 10x larger Llama3-70b-instruct in generating human-preferred summaries, demonstrating that smaller models can achieve superior performance with appropriate training. The full dataset and SummLlama3-8B model are available at https://huggingface.co/datasets/DISLab/FeedSum and https://huggingface.co/DISLab/SummLlama3-8B.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 45 of its 59 references that are in the map. First posted on arXiv 17 Oct 2024.

Builds on

45 of 59 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024AlignSum: Data Pyramid Hierarchical Fine-tuning for Aligning with Human Summarization Preference1 Oct
  2. 2024BiasDPO: Mitigating Bias in Language Models through Direct Preference Optimization18 Jul
  3. 2023Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Merging17 Oct
  4. 2022On Improving Summarization Factual Consistency from Natural Language Feedback20 Dec
  5. 2020Learning to summarize from human feedback2 Sep

Other references · 40

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  6. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  7. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  8. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  9. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  10. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,102
  11. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 900
  12. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 812
28 more
  1. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 625
  2. 2019Text Summarization with Pretrained Encoderscited by 556
  3. 2018A Discourse-Aware Attention Model for Abstractive Summarization of Long Documentscited by 478
  4. 2023Is ChatGPT a Good NLG Evaluator? A Preliminary Studycited by 422
  5. 2021SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarizationcited by 407
  6. 2024Hallucination of Multimodal Large Language Models: A Surveycited by 388
  7. 2021Efficient Attentions for Long Document Summarizationcited by 345
  8. 2021Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality Metricscited by 287
  9. 2022Towards a Unified Multi-Dimensional Evaluator for Text Generationcited by 264
  10. 2024MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documentscited by 257
  11. 2021QAFactEval: Improved QA-Based Factual Consistency Evaluation for Summarizationcited by 236
  12. 2018WikiHow: A Large Scale Text Summarization Datasetcited by 204
  13. 2021DialogSum: A Real-Life Scenario Dialogue Summarization Datasetcited by 183
  14. 2020Re-evaluating Evaluation in Text Summarizationcited by 145
  15. 2024Controllable Preference Optimization: Toward Controllable Multi-Objective Alignmentcited by 107
  16. 2021MediaSum: A Large-scale Media Interview Dataset for Dialogue Summarizationcited by 104
  17. 2023MeetingBank: A Benchmark Dataset for Meeting Summarizationcited by 84
  18. 2023Factually Consistent Summarization via Reinforcement Learning with Textual Entailment Feedbackcited by 72
  19. 2024TofuEval: Evaluating Hallucinations of LLMs on Topic-Focused Dialogue Summarizationcited by 66
  20. 2023Language Model Alignment with Elastic Resetcited by 34
  21. 2023Preference-grounded Token-level Guidance for Language Model Fine-tuningcited by 31
  22. 2024Preference Tuning For Toxicity Mitigation Generalizes Across Languagescited by 31
  23. 2024SPO: Multi-Dimensional Preference Sequential Alignment With Implicit Reward Modelingcited by 21
  24. 2024UniSumEval: Towards Unified, Fine-Grained, Multi-Dimensional Summarization Evaluation for LLMscited by 21
  25. 2021Attention Temperature Matters in Abstractive Summarization Distillationcited by 14
  26. 2024SYNFAC-EDIT: Synthetic Imitation Edit Feedback for Factual Alignment in Clinical Summarizationcited by 12
  27. 2024Applying RLAIF for Code Generation with API-usage in Lightweight LLMscited by 10
  28. 2023Enhancing Abstractiveness of Summarization Models through Calibrated Distillationcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 13; 2026: 10.

24 papers.

Sort:

By area:

  1. 2024EXIT: Context-Aware Extractive Compression for Enhancing Retrieval-Augmented Generationcited by 17
  2. 2025PIIvot: A Lightweight NLP Anonymization Framework for Question-Anchored Tutoring Dialoguescited by 11
  3. 2025Learning to summarize user information for personalized reinforcement learning from human feedbackcited by 10
  4. 2025The Bias is in the Details: An Assessment of Cognitive Bias in LLMscited by 10
  5. 2025Hallucinate at the Last in Long Response Generation: A Case Study on Long Document Summarizationcited by 6
  6. 2025Unraveling Misinformation Propagation in LLM Reasoningcited by 6
  7. 2025Flex-Judge: Text-Only Reasoning Unleashes Zero-Shot Multimodal Evaluatorscited by 6
  8. 2025ReFeed: Multi-dimensional Summarization Refinement with Reflective Reasoning on Feedbackcited by 4
  9. 2026FeedEval: Pedagogically Aligned Evaluation of LLM-Generated Essay Feedbackcited by 4
  10. 2026Learning to Control Summaries with Score Rankingcited by 4
  11. 2026Calibrating Model-Based Evaluation Metrics for Summarizationcited by 3
  12. 2025Aligning Extraction and Generation for Robust Retrieval-Augmented Generationcited by 2
12 more
  1. 2025Adaptive Planning for Multi-Attribute Controllable Summarization with Monte Carlo Tree Searchcited by 2
  2. 2025What Are They Talking About? A Benchmark of Knowledge-Grounded Discussion Summarizationcited by 1
  3. 2026Alignment Tuning for Large Language Models: A Data-Centric Lens on Alignment Data Pipelinescited by 1
  4. 2026A Large-Scale Multi-Dimensional Empirical Study of LLMs for Conversation Summarizationcited by 1
  5. 2025References Indeed Matter? Reference-Free Preference Optimization for Conversational Query Reformulationcited by 0
  6. 2025MARCO: Meta-Reflection with Cross-Referencing for Code Reasoningcited by 0
  7. 2025Keystep Recognition using Graph Neural Networkscited by 0
  8. 2026DIAL-SUMMER: A Structured Evaluation Framework of Hierarchical Errors in Dialogue Summariescited by 0
  9. 2026Beyond Overlap Metrics: Rewarding Reasoning and Preferences for Faithful Multi-Role Dialogue Summarizationcited by 0
  10. 2026LLM-ReSum: A Framework for LLM Reflective Summarization through Self-Evaluationcited by 0
  11. 2026Distilling Long-CoT Reasoning through Collaborative Step-wise Multi-Teacher Decodingcited by 0
  12. 2026Ready2Blend: From Natural-Language Instructions to Composable Alignment Promptscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Learning from Self Critique and Refinement for Faithful LLM Summarization
  2. 2024Model-based Preference Optimization in Abstractive Summarization without Human Feedback
  3. 2024Learning to Verify Summary Facts with Fine-Grained LLM Feedback
  4. 2022Make The Most of Prior Data: A Solution for Interactive Text Summarization with Preference Feedback
  5. 2023LLMRefine: Pinpointing and Refining Large Language Models via Fine-Grained Actionable Feedback
  6. 2025Towards Multi-dimensional Evaluation of LLM Summarization across Domains and Languages
  7. 2024How Much Annotation is Needed to Compare Summarization Models?
  8. 2023Large Language Models are Not Yet Human-Level Evaluators for Abstractive Summarization