# ACLSum: A New Dataset for Aspect-based Summarization of Scientific Publications

arXiv: https://arxiv.org/abs/2403.05303 · HTML: https://mlsanity.com/paper/2403.05303/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 54 references that are in the map.
- First posted on arXiv 8 Mar 2024.

## Builds on (most cited first, 30 of 36)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [SummEval: Re-evaluating Summarization Evaluation](https://mlsanity.com/paper/2007.12626/) (2020)
- [Text Summarization with Pretrained Encoders](https://mlsanity.com/paper/1908.08345/) (2019)
- [A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents](https://mlsanity.com/paper/1804.05685/) (2018)
- [Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models](https://mlsanity.com/paper/2108.08877/) (2021)
- [Benchmarking Large Language Models for News Summarization](https://mlsanity.com/paper/2301.13848/) (2023)
- [Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategies](https://mlsanity.com/paper/1804.11283/) (2018)
- [SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documents](https://mlsanity.com/paper/1611.04230/) (2016)
- [Neural Text Summarization: A Critical Evaluation](https://mlsanity.com/paper/1908.08960/) (2019)
- [BRIO: Bringing Order to Abstractive Summarization](https://mlsanity.com/paper/2203.16804/) (2022)
- [TLDR: Extreme Summarization of Scientific Documents](https://mlsanity.com/paper/2004.15011/) (2020)
- [BLEU might be Guilty but References are not Innocent](https://mlsanity.com/paper/2004.06063/) (2020)
- [Abstractive Summarization of Reddit Posts with Multi-level Memory Networks](https://mlsanity.com/paper/1811.00783/) (2018)
- [Towards Question-Answering as an Automatic Metric for Evaluating the Content Quality of a Summary](https://mlsanity.com/paper/2010.00490/) (2020)
- [On Extractive and Abstractive Neural Document Summarization with Transformer Language Models](https://mlsanity.com/paper/1909.03186/) (2019)
- [An Empirical Survey on Long Document Summarization: Datasets, Models and Metrics](https://mlsanity.com/paper/2207.00939/) (2022)
- [Growth rates of modern science: A bibliometric analysis based on the number of publications and cited references](https://mlsanity.com/paper/1402.4578/) (2014)
- [WikiAsp: A Dataset for Multi-domain Aspect-based Summarization](https://mlsanity.com/paper/2011.07832/) (2020)
- [On the Limitations of Reference-Free Evaluations of Generated Text](https://mlsanity.com/paper/2210.12563/) (2022)
- [A Unified Model for Extractive and Abstractive Summarization using Inconsistency Loss](https://mlsanity.com/paper/1805.06266/) (2018)
- [DYLE: Dynamic Latent Extraction for Abstractive Long-Input Summarization](https://mlsanity.com/paper/2110.08168/) (2021)

## Built on by (most cited first, 20 of 20)

- [Is It Really Long Context if All You Need Is Retrieval? Towards Genuinely Difficult Long Context NLP](https://mlsanity.com/paper/2407.00402/) (2024)
- [MASSW: A New Dataset and Benchmark Tasks for AI-Assisted Scientific Workflows](https://mlsanity.com/paper/2406.06357/) (2024)
- [MultiFinBen: Benchmarking Large Language Models for Multilingual and Multimodal Financial Application](https://mlsanity.com/paper/2506.14028/) (2025)
- [An Evaluation of Estimative Uncertainty in Large Language Models](https://mlsanity.com/paper/2405.15185/) (2024)
- [What Is That Talk About? A Video-to-Text Summarization Dataset for Scientific Presentations](https://mlsanity.com/paper/2502.08279/) (2025)
- [PosterSum: A Multimodal Benchmark for Scientific Poster Summarization](https://mlsanity.com/paper/2502.17540/) (2025)
- [The Power of Personality: A Human Simulation Perspective to Investigate Large Language Model Agents](https://mlsanity.com/paper/2502.20859/) (2025)
- [TracSum: A New Benchmark for Aspect-Based Summarization with Sentence-Level Traceability in Medical Domain](https://mlsanity.com/paper/2508.13798/) (2025)
- [AUTALIC: A Dataset for Anti-AUTistic Ableist Language In Context](https://mlsanity.com/paper/2410.16520/) (2024)
- [The State and Fate of Summarization Datasets: A Survey](https://mlsanity.com/paper/2411.04585/) (2024)
- [NUTSHELL: A Dataset for Abstract Generation from Scientific Talks](https://mlsanity.com/paper/2502.16942/) (2025)
- [Efficient Layer-wise LLM Fine-tuning for Revision Intention Prediction](https://mlsanity.com/paper/2510.00268/) (2025)
- [Aspect-Oriented Summarization for Psychiatric Short-Term Readmission Prediction](https://mlsanity.com/paper/2502.10388/) (2025)
- [ShapE-GRPO: Shapley-Enhanced Reward Allocation for Multi-Candidate LLM Training](https://mlsanity.com/paper/2603.29871/) (2026)
- [Can Smaller LLMs do better? Unlocking Cross-Domain Potential through Parameter-Efficient Fine-Tuning for Text Summarization](https://mlsanity.com/paper/2509.01314/) (2025)
- [LNE-Blocking: An Efficient Framework for Contamination Mitigation Evaluation on Large Language Models](https://mlsanity.com/paper/2509.15218/) (2025)
- [CARPAS: Towards Content-Aware Refinement of Provided Aspects for Summarization in Large Language Models](https://mlsanity.com/paper/2510.07177/) (2025)
- [MAPLE: Medical Aspect-Based Summarization with Phrase-Level Evidence](https://mlsanity.com/paper/2601.03418/) (2026)
- [Intention-Adaptive LLM Fine-Tuning for Text Revision Generation](https://mlsanity.com/paper/2602.00477/) (2026)
- [SciZoom: A Large-scale Benchmark for Hierarchical Scientific Summarization across the LLM Era](https://mlsanity.com/paper/2603.16131/) (2026)

## Abstract (arXiv)

Extensive efforts in the past have been directed toward the development of summarization datasets. However, a predominant number of these resources have been (semi)-automatically generated, typically through web data crawling, resulting in subpar resources for training and evaluating summarization systems, a quality compromise that is arguably due to the substantial costs associated with generating ground-truth summaries, particularly for diverse languages and specialized domains. To address this issue, we present ACLSum, a novel summarization dataset carefully crafted and evaluated by domain experts. In contrast to previous datasets, ACLSum facilitates multi-aspect summarization of scientific papers, covering challenges, approaches, and outcomes in depth. Through extensive experiments, we evaluate the quality of our resource and the performance of models based on pretrained language models and state-of-the-art large language models (LLMs). Additionally, we explore the effectiveness of extractive versus abstractive summarization within the scholarly domain on the basis of automatically discovered aspects. Our results corroborate previous findings in the general domain and indicate the general superiority of end-to-end aspect-based summarization. Our data is released at https://github.com/sobamchan/aclsum.