# Disordered-DABS: A Benchmark for Dynamic Aspect-Based Summarization in Disordered Texts

arXiv: https://arxiv.org/abs/2402.10554 · HTML: https://mlsanity.com/paper/2402.10554/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 29 references that are in the map.
- First posted on arXiv 16 Feb 2024.

## Builds on (most cited first, 19 of 19)

- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [SummEval: Re-evaluating Summarization Evaluation](https://mlsanity.com/paper/2007.12626/) (2020)
- [QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization](https://mlsanity.com/paper/2104.05938/) (2021)
- [WikiHow: A Large Scale Text Summarization Dataset](https://mlsanity.com/paper/1810.09305/) (2018)
- [Exploring the Limits of ChatGPT for Query or Aspect-based Text Summarization](https://mlsanity.com/paper/2302.08081/) (2023)
- [Extractive Opinion Summarization in Quantized Transformer Spaces](https://mlsanity.com/paper/2012.04443/) (2020)
- [WikiAsp: A Dataset for Multi-domain Aspect-based Summarization](https://mlsanity.com/paper/2011.07832/) (2020)
- [SQuALITY: Building a Long-Document Summarization Dataset the Hard Way](https://mlsanity.com/paper/2205.11465/) (2022)
- [Aspect-Controllable Opinion Summarization](https://mlsanity.com/paper/2109.03171/) (2021)
- [ASPECTNEWS: Aspect-Oriented Summarization of News Documents](https://mlsanity.com/paper/2110.08296/) (2021)
- [Bringing Structure into Summaries: a Faceted Summarization Dataset for Long Scientific Documents](https://mlsanity.com/paper/2106.00130/) (2021)
- [Generating Query Focused Summaries from Query-Free Resources](https://mlsanity.com/paper/2012.14774/) (2020)
- [OASum: Large-Scale Open Domain Aspect-based Summarization](https://mlsanity.com/paper/2212.09233/) (2022)
- [Summarizing Text on Any Aspects: A Knowledge-Informed Weakly-Supervised Approach](https://mlsanity.com/paper/2010.06792/) (2020)
- [EntSUM: A Data Set for Entity-Centric Summarization](https://mlsanity.com/paper/2204.02213/) (2022)
- [OpenAsp: A Benchmark for Multi-document Open Aspect-based Summarization](https://mlsanity.com/paper/2312.04440/) (2023)

## Built on by (most cited first, 5 of 5)

- [BenchMarker: An Education-Inspired Toolkit for Highlighting Flaws in Multiple-Choice Benchmarks](https://mlsanity.com/paper/2602.06221/) (2026)
- [TracSum: A New Benchmark for Aspect-Based Summarization with Sentence-Level Traceability in Medical Domain](https://mlsanity.com/paper/2508.13798/) (2025)
- [MODABS: Multi-Objective Learning for Dynamic Aspect-Based Summarization](https://mlsanity.com/paper/2406.03479/) (2024)
- [CARPAS: Towards Content-Aware Refinement of Provided Aspects for Summarization in Large Language Models](https://mlsanity.com/paper/2510.07177/) (2025)
- [Structured Claim-Level Discourse Representations for Dense Health Narratives](https://mlsanity.com/paper/2609.18905/) (2026)

## Abstract (arXiv)

Aspect-based summarization has seen significant advancements, especially in structured text. Yet, summarizing disordered, large-scale texts, like those found in social media and customer feedback, remains a significant challenge. Current research largely targets predefined aspects within structured texts, neglecting the complexities of dynamic and disordered environments. Addressing this gap, we introduce Disordered-DABS, a novel benchmark for dynamic aspect-based summarization tailored to unstructured text. Developed by adapting existing datasets for cost-efficiency and scalability, our comprehensive experiments and detailed human evaluations reveal that Disordered-DABS poses unique challenges to contemporary summarization models, including state-of-the-art language models such as GPT-3.5.