# TracSum: A New Benchmark for Aspect-Based Summarization with Sentence-Level Traceability in Medical Domain

arXiv: https://arxiv.org/abs/2508.13798 · HTML: https://mlsanity.com/paper/2508.13798/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 53 references that are in the map.
- First posted on arXiv 19 Aug 2025.

## Builds on (most cited first, 24 of 24)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [Gemma 3 Technical Report](https://mlsanity.com/paper/2503.19786/) (2025)
- [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://mlsanity.com/paper/2305.14251/) (2023)
- [When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories](https://mlsanity.com/paper/2212.10511/) (2022)
- [Enabling Large Language Models to Generate Text with Citations](https://mlsanity.com/paper/2305.14627/) (2023)
- [A Neural Attention Model for Abstractive Sentence Summarization](https://mlsanity.com/paper/1509.00685/) (2015)
- [A Deep Reinforced Model for Abstractive Summarization](https://mlsanity.com/paper/1705.04304/) (2017)
- [QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization](https://mlsanity.com/paper/2104.05938/) (2021)
- [Evaluating Verifiability in Generative Search Engines](https://mlsanity.com/paper/2304.09848/) (2023)
- [Neural Summarization by Extracting Sentences and Words](https://mlsanity.com/paper/1603.07252/) (2016)
- [M-RAG: Reinforcing Large Language Model Performance through Retrieval-Augmented Generation with Multiple Partitions](https://mlsanity.com/paper/2405.16420/) (2024)
- [OASum: Large-Scale Open Domain Aspect-based Summarization](https://mlsanity.com/paper/2212.09233/) (2022)
- [ACLSum: A New Dataset for Aspect-based Summarization of Scientific Publications](https://mlsanity.com/paper/2403.05303/) (2024)
- [FactPICO: Factuality Evaluation for Plain Language Summarization of Medical Evidence](https://mlsanity.com/paper/2402.11456/) (2024)
- [Salience Allocation as Guidance for Abstractive Summarization](https://mlsanity.com/paper/2210.12330/) (2022)
- [HydraSum: Disentangling Stylistic Features in Text Summarization using Multi-Decoder Models](https://mlsanity.com/paper/2110.04400/) (2021)
- [FaMeSumm: Investigating and Improving Faithfulness of Medical Summarization](https://mlsanity.com/paper/2311.02271/) (2023)
- [A Two-Phase Approach for Abstractive Podcast Summarization](https://mlsanity.com/paper/2011.08291/) (2020)
- [Disordered-DABS: A Benchmark for Dynamic Aspect-Based Summarization in Disordered Texts](https://mlsanity.com/paper/2402.10554/) (2024)
- [Traceable Text: Deepening Reading of AI-Generated Summaries with Phrase-Level Provenance Links](https://mlsanity.com/paper/2409.13099/) (2024)
- [Supporting Evidence-Based Medicine by Finding Both Relevant and Significant Works](https://mlsanity.com/paper/2407.18383/) (2024)

## Built on by (most cited first, 4 of 4)

- [MAPLE: Medical Aspect-Based Summarization with Phrase-Level Evidence](https://mlsanity.com/paper/2601.03418/) (2026)
- [eTracer: Polarity-Aware Evidence Grounding](https://mlsanity.com/paper/2601.03669/) (2026)
- [BLUEmed: Retrieval-Augmented Multi-Agent Debate for Clinical Error Detection](https://mlsanity.com/paper/2604.10389/) (2026)
- [LOOMSUM: Weaving Quantitative and Narrative Evidence for Faithful Long Text-Table Summarization](https://mlsanity.com/paper/2609.00241/) (2026)

## Abstract (arXiv)

While document summarization with LLMs has enhanced access to textual information, concerns about the factual accuracy of these summaries persist, especially in the medical domain. Tracing evidence from which summaries are derived enables users to assess their accuracy, thereby alleviating this concern. In this paper, we introduce TracSum, a novel benchmark for traceable, aspect-based summarization, in which generated summaries are paired with sentence-level citations, enabling users to trace back to the original context. First, we annotate 500 medical abstracts for seven key medical aspects, yielding 3.5K summary-citation pairs. We then propose a fine-grained evaluation framework for this new task, designed to assess the completeness and consistency of generated content using four metrics. Finally, we introduce a summarization pipeline, Track-Then-Sum, which serves as a baseline method for comparison. In experiments, we evaluate both this baseline and a set of LLMs on TracSum, and conduct a human evaluation to assess the evaluation results. The findings demonstrate that TracSum can serve as an effective benchmark for traceable, aspect-based summarization tasks. We also observe that explicitly performing sentence-level tracking prior to summarization enhances generation accuracy, while incorporating the full context further improves completeness.