Improving Argument Saliency Coverage in Small LLMs for Long Legal Opinion Summarization via Sequence-Level Distillation
On the map: Knowledge distillation → Knowledge distillation for LLMs
Abstract (arXiv)
We show that sequence-level distillation from a capable long-context teacher model is a simple, annotation-free, and data-efficient strategy for improving argument saliency coverage in long legal opinion summarization, where small LLMs often struggle to retain the most salient argumentative content. Across student model sizes, distillation consistently surpasses tuning on expert-written summaries in our legal-opinion setting. We further demonstrate that most gains are achieved with as few as ~10 training summaries, highlighting the strong data efficiency of teacher-generated supervision. Finally, we find that summary distillation is sufficient for improvements: reasoning-chain distillation remains competitive with summary-only distillation, but provides marginal benefit when combined with summary supervision.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Knowledge distillation for LLMs, one of 12 directions in Knowledge distillation. Builds on 20 of its 33 references that are in the map. First posted on arXiv 30 Aug 2026.
Builds on
20 of 33 references.
Closest ancestors · 1
Same direction, newest first
Other references · 19
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
- 2025s1: Simple test-time scalingcited by 1,341
- 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,317
- 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,282
- 2015Teaching Machines to Read and Comprehendcited by 1,049
- 2016Sequence-Level Knowledge Distillationcited by 801
- 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 737
- 2024MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documentscited by 257
- 2024How to Train Long-Context Language Models (Effectively)cited by 146
- 2023From Sparse to Dense: GPT-4 Summarization with Chain of Density Promptingcited by 52
7 more
- 2022ArgLegalSumm: Improving Abstractive Summarization of Legal Documents with Argument Miningcited by 25
- 2025Behavioral Analysis of Information Salience in Large Language Modelscited by 12
- 2024TriSum: Learning Summarization Ability from Large Language Models with Structured Rationalecited by 10
- 2023Towards Argument-Aware Abstractive Summarization of Long Legal Opinions with Summary Rerankingcited by 9
- 2025Discourse-Driven Evaluation: Unveiling Factual Inconsistency in Long Document Summarizationcited by 8
- 2024CaseSumm: A Large-Scale Dataset for Long-Context Summarization from U.S. Supreme Court Opinionscited by 5
- 2025ARC: Argument Representation and Coverage Analysis for Zero-Shot Long Document Summarization with Instruction Following LLMscited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Arg-LLaDA: Argument Summarization via Large Language Diffusion Models and Sufficiency-Aware Refinement
- 2025Modeling Motivated Reasoning in Law: Evaluating Strategic Role Conditioning in LLM Summarization
- 2026BenGER: Benchmarking LLM Systems on Subsumption-Based Legal Reasoning in German Law
- 2026LAMUS: A Large-Scale Corpus for Legal Argument Mining from U.S. Caselaw using LLMs
- 2026Gavel: Agent Meets Checklist for Evaluating LLMs on Long-Context Legal Summarization
- 2026Multi-Legal-Bench: Evaluating LLMs on Legal Reasoning Across Jurisdictions, Languages, and Legal Traditions
- 2025LEXam: Benchmarking Legal Reasoning on 340 Law Exams
- 2026A Tree-of-Thoughts Inspired Hybrid Approach for Legal Case Judgement Summarization using LLMs