mlsanityData to 14 Oct 2026

Language Model as an Annotator: Exploring DialoGPT for Dialogue Summarization

arXiv 2105.12544 ↗First posted 26 May 2021cs.CLcited by 46

Not on the map. Why?

Abstract (arXiv)

Current dialogue summarization systems usually encode the text with a number of general semantic features (e.g., keywords and topics) to gain more powerful dialogue modeling capabilities. However, these features are obtained via open-domain toolkits that are dialog-agnostic or heavily relied on human annotations. In this paper, we show how DialoGPT, a pre-trained model for conversational response generation, can be developed as an unsupervised dialogue annotator, which takes advantage of dialogue background knowledge encoded in DialoGPT. We apply DialoGPT to label three types of features on two dialogue summarization datasets, SAMSum and AMI, and employ pre-trained and non pre-trained models as our summarizes. Experimental results show that our proposed method can obtain remarkable improvements on both datasets and achieves new state-of-the-art performance on the SAMSum dataset.

Key facts. Cited by 46 papers in the map (data through 14 Oct 2026). Builds on 30 of its 50 references that are in the map. First posted on arXiv 26 May 2021.

Builds on

30 of 50 references.

References · 30

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  6. 2016Layer Normalizationcited by 5,669
  7. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  8. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,317
  9. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,292
  10. 2019How Can We Know What Language Models Know?cited by 959
  11. 2015Pointer Networkscited by 895
  12. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
18 more
  1. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 812
  2. 2020Stanza: A Python Natural Language Processing Toolkit for Many Human Languagescited by 723
  3. 2019Text Summarization with Pretrained Encoderscited by 556
  4. 2019SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarizationcited by 535
  5. 2016SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documentscited by 246
  6. 2020Data Augmentation using Pre-trained Transformer Modelscited by 168
  7. 2020Birds have four legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre-trained Language Modelscited by 110
  8. 2019A Stack-Propagation Framework with Token-Level Intent Detection for Spoken Language Understandingcited by 107
  9. 2020Language Models are Open Knowledge Graphscited by 95
  10. 2020Multi-View Sequence-to-Sequence Models with Conversational Structure for Abstractive Dialogue Summarizationcited by 72
  11. 2018Unsupervised Abstractive Meeting Summarization with Multi-Sentence Compression and Budgeted Submodular Maximizationcited by 46
  12. 2018Abstractive Dialogue Summarization with Sentence-Gated Modeling Optimized by Dialogue Actscited by 44
  13. 2019Topic-aware Pointer-Generator Networks for Summarizing Spoken Conversationscited by 41
  14. 2020Co-GAT: A Co-Interactive Graph Attention Network for Joint Dialog Act Recognition and Sentiment Classificationcited by 29
  15. 2020Dialogue Discourse-Aware Graph Model and Data Augmentation for Meeting Summarizationcited by 28
  16. 2020Incorporating Commonsense Knowledge into Abstractive Dialogue Summarization via Heterogeneous Graph Networkscited by 22
  17. 2020How Domain Terminology Affects Meeting Summarization Performancecited by 14
  18. 2021Ask2Transformers: Zero-Shot Domain labelling with Pre-trained Language Modelscited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 5; 2022: 14; 2023: 13; 2024: 9; 2025: 4; 2026: 1.

46 papers.

Sort:

By area:

  1. 2022Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluationcited by 142
  2. 2022Pretrained Language Models for Text Generation: A Surveycited by 122
  3. 2024A Comprehensive Survey on Process-Oriented Automatic Text Summarization with Exploration of LLM-Based Methodscited by 102
  4. 2023LLMaAA: Making Large Language Models as Active Annotatorscited by 79
  5. 2022An Empirical Survey on Long Document Summarization: Datasets, Models and Metricscited by 74
  6. 2021DialogLM: Pre-trained Model for Long Dialogue Understanding and Summarizationcited by 72
  7. 2021A Survey on Dialogue Summarization: Recent Advances and New Frontierscited by 51
  8. 2021CONFIT: Toward Faithful Dialogue Summarization with Linguistically-Informed Contrastive Fine-tuningcited by 44
  9. 2021Controllable Neural Dialogue Summarization with Personal Named Entity Planningcited by 36
  10. 2023What Matters In The Structured Pruning of Generative Language Models?cited by 30
  11. 2022ClidSum: A Benchmark Dataset for Cross-Lingual Dialogue Summarizationcited by 25
  12. 2024TELEClass: Taxonomy Enrichment and LLM-Enhanced Hierarchical Text Classification with Minimal Supervisioncited by 25
34 more
  1. 2022Dialogue Summaries as Dialogue States (DS2), Template-Guided Summarization for Few-shot Dialogue State Trackingcited by 21
  2. 2022Improving the Faithfulness of Abstractive Summarization via Entity Coverage Controlcited by 21
  3. 2022Counseling Summarization using Mental Health Knowledge Guided Utterance Filteringcited by 18
  4. 2023VCSUM: A Versatile Chinese Meeting Summarization Datasetcited by 18
  5. 2024Text Generation: A Systematic Literature Review of Tasks, Evaluation, and Challengescited by 18
  6. 2024Towards LifeSpan Cognitive Systemscited by 13
  7. 2021Response Generation with Context-Aware Prompt Learningcited by 12
  8. 2024Knowledge Planning in Large Language Models for Domain-Aligned Counseling Summarizationcited by 11
  9. 2023SuperDialseg: A Large-scale Dataset for Supervised Dialogue Segmentationcited by 10
  10. 2023Annotating and Detecting Fine-grained Factual Errors for Dialogue Summarizationcited by 10
  11. 2022Abstractive Meeting Summarization: A Surveycited by 9
  12. 2023Towards Personalized Review Summarization by Modeling Historical Reviews from Customer and Product Separatelycited by 9
  13. 2022The Cross-lingual Conversation Summarization Challengecited by 8
  14. 2022Back to the Future: Bidirectional Information Decoupling Network for Multi-turn Dialogue Modelingcited by 7
  15. 2023Revisiting Cross-Lingual Summarization: A Corpus-based Study and A New Benchmark with Improved Annotationcited by 7
  16. 2023Multi-Stage Pre-training Enhanced by ChatGPT for Multi-Scenario Multi-Domain Dialogue Summarizationcited by 7
  17. 2024Baichuan2-Sum: Instruction Finetune Baichuan2-7B Model for Dialogue Summarizationcited by 7
  18. 2025Multi-Party Conversational Agents: A Surveycited by 7
  19. 2023Enhancing Dialogue Summarization with Topic-Aware Global- and Local- Level Centralitycited by 5
  20. 2023Large Language Models are Few-Shot Training Example Generators: A Case Study in Fallacy Recognitioncited by 5
  21. 2025When Memory Becomes a Vulnerability: Towards Multi-turn Jailbreak Attacks against Text-to-Image Generation Systemscited by 5
  22. 2022Post-Training Dialogue Summarization using Pseudo-Paraphrasingcited by 4
  23. 2022Effectiveness of French Language Models on Abstractive Dialogue Summarization Taskcited by 3
  24. 2022Taxonomy of Abstractive Dialogue Summarization: Scenarios, Approaches and Future Directionscited by 3
  25. 2022KPT: Keyword-guided Pre-training for Grounded Dialog Generationcited by 3
  26. 2023Private Meeting Summarization Without Performance Losscited by 3
  27. 2023Responsible AI Considerations in Text Summarization Research: A Review of Current Practicescited by 2
  28. 2024Text2Traj2Text: Learning-by-Synthesis Framework for Contextual Captioning of Human Movement Trajectoriescited by 1
  29. 2025CAuSE: Decoding Multimodal Classifiers using Faithful Natural Language Explanationcited by 1
  30. 2025Topic Segmentation Using Generative Language Modelscited by 1
  31. 2026Right Reset: Chunking by Prefix Removalcited by 1
  32. 2023Reconstruct Before Summarize: An Efficient Two-Step Framework for Condensing and Summarizing Meeting Transcriptscited by 0
  33. 2024SumRec: A Framework for Recommendation using Open-Domain Dialoguecited by 0
  34. 2024Systematic Exploration of Dialogue Summarization Approaches for Reproducibility, Comparative Assessment, and Methodological Innovations for Advancing Natural Language Processing in Abstractive Summarizationcited by 0