QUARTZ : QA-based Unsupervised Abstractive Refinement for Task-oriented Dialogue Summarization
Not on the map. Why?
Abstract (arXiv)
Dialogue summarization aims to distill the core meaning of a conversation into a concise text. This is crucial for reducing the complexity and noise inherent in dialogue-heavy applications. While recent approaches typically train language models to mimic human-written summaries, such supervision is costly and often results in outputs that lack task-specific focus limiting their effectiveness in downstream applications, such as medical tasks. In this paper, we propose app, a framework for task-oriented utility-based dialogue summarization. app starts by generating multiple summaries and task-oriented question-answer pairs from a dialogue in a zero-shot manner using a pool of large language models (LLMs). The quality of the generated summaries is evaluated by having LLMs answer task-related questions before textit(i) selecting the best candidate answers and textit(ii) identifying the most informative summary based on these answers. Finally, we fine-tune the best LLM on the selected summaries. When validated on multiple datasets, app demonstrates its effectiveness by achieving competitive results in various zero-shot settings, rivaling fully-supervised State-of-the-Art (SotA) methods.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 29 of its 68 references that are in the map. First posted on arXiv 30 Sep 2025.
Builds on
29 of 68 references.
References · 29
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
- 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
- 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
- 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,210
- 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
- 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 686
- 2019SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarizationcited by 535
17 more
- 2021A Survey of Data Augmentation Approaches for NLPcited by 409
- 2023Adapted Large Language Models Can Outperform Medical Experts in Clinical Text Summarizationcited by 246
- 2024LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Taskscited by 243
- 2021DialogSum: A Real-Life Scenario Dialogue Summarization Datasetcited by 183
- 2023Does Synthetic Data Generation of LLMs Help Clinical Text Mining?cited by 154
- 2024Pride and Prejudice: LLM Amplifies Self-Bias in Self-Refinementcited by 114
- 2025Multiagent Finetuning: Self Improvement with Diverse Reasoning Chainscited by 95
- 2024FineSurE: Fine-grained Summarization Evaluation using LLMscited by 83
- 2021Medically Aware GPT-3 as a Data Generator for Medical Dialogue Summarizationcited by 76
- 2022POTATO: The Portable Text Annotation Toolcited by 64
- 2019Generalized Data Augmentation for Low-Resource Translationcited by 53
- 2020Synthetic Data Augmentation for Zero-Shot Cross-Lingual Question Answeringcited by 41
- 2021TWEETSUMM -- A Dialog Summarization Dataset for Customer Servicecited by 34
- 2019Data Augmentation for BERT Fine-Tuning in Open-Domain Question Answeringcited by 21
- 2021Low-Resource Dialogue Summarization with Domain-Agnostic Multi-Source Pretrainingcited by 19
- 2021TODSum: Task-Oriented Dialogue Summarization with State Trackingcited by 19
- 2021Summary Grounded Conversation Generationcited by 5
Built on by
Citing papers per year: 2026: 2.
2 papers.