# Uncovering the Potential of ChatGPT for Discourse Analysis in Dialogue: An Empirical Study

arXiv: https://arxiv.org/abs/2305.08391 · HTML: https://mlsanity.com/paper/2305.08391/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 60 references that are in the map.
- First posted on arXiv 15 May 2023.

## Builds on (most cited first, 30 of 36)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [Personalizing Dialogue Agents: I have a dog, do you have pets too?](https://mlsanity.com/paper/1801.07243/) (2018)
- [MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling](https://mlsanity.com/paper/1810.00278/) (2018)
- [DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset](https://mlsanity.com/paper/1710.03957/) (2017)
- [Capabilities of GPT-4 on Medical Challenge Problems](https://mlsanity.com/paper/2303.13375/) (2023)
- [Is ChatGPT a General-Purpose Natural Language Processing Task Solver?](https://mlsanity.com/paper/2302.06476/) (2023)
- [Benchmarking Large Language Models for News Summarization](https://mlsanity.com/paper/2301.13848/) (2023)
- [ChatIE: Zero-Shot Information Extraction via Chatting with ChatGPT](https://mlsanity.com/paper/2302.10205/) (2023)
- [Leveraging Large Language Models for Multiple Choice Question Answering](https://mlsanity.com/paper/2210.12353/) (2022)
- [Is ChatGPT A Good Translator? Yes With GPT-4 As The Engine](https://mlsanity.com/paper/2301.08745/) (2023)
- [The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems](https://mlsanity.com/paper/1506.08909/) (2015)
- [Key-Value Retrieval Networks for Task-Oriented Dialogue](https://mlsanity.com/paper/1705.05414/) (2017)
- [Exploring the Limits of ChatGPT for Query or Aspect-based Text Summarization](https://mlsanity.com/paper/2302.08081/) (2023)
- [Is ChatGPT a Good Sentiment Analyzer? A Preliminary Study](https://mlsanity.com/paper/2304.04339/) (2023)
- [Exploring the Feasibility of ChatGPT for Event Extraction](https://mlsanity.com/paper/2303.03836/) (2023)
- [Molweni: A Challenge Multiparty Dialogues-based Machine Reading Comprehension Dataset with Discourse Structure](https://mlsanity.com/paper/2004.05080/) (2020)
- [Extractive Summarization via ChatGPT for Faithful Summary Generation](https://mlsanity.com/paper/2304.04193/) (2023)
- [Zero-shot Temporal Relation Extraction with ChatGPT](https://mlsanity.com/paper/2304.05454/) (2023)
- [Topic-Aware Multi-turn Dialogue Modeling](https://mlsanity.com/paper/2009.12539/) (2020)
- [ChatGPT Evaluation on Sentence Level Relations: A Focus on Temporal, Causal, and Discourse Relations](https://mlsanity.com/paper/2304.14827/) (2023)
- [A Deep Sequential Model for Discourse Parsing on Multi-Party Dialogues](https://mlsanity.com/paper/1812.00176/) (2018)
- [NaturalConv: A Chinese Dialogue Dataset Towards Multi-turn Topic-driven Conversation](https://mlsanity.com/paper/2103.02548/) (2021)
- [Unsupervised Topic Segmentation of Meetings with BERT Embeddings](https://mlsanity.com/paper/2106.12978/) (2021)
- [Improving Unsupervised Dialogue Topic Segmentation with Utterance-Pair Coherence Scoring](https://mlsanity.com/paper/2106.06719/) (2021)
- [Applying BERT and ChatGPT for Sentiment Analysis of Lyme Disease in Scientific Literature](https://mlsanity.com/paper/2302.06474/) (2023)
- [TIAGE: A Benchmark for Topic-Shift Aware Dialog Modeling](https://mlsanity.com/paper/2109.04562/) (2021)
- [Improving Context Modeling in Neural Topic Segmentation](https://mlsanity.com/paper/2010.03138/) (2020)
- [Improving Multi-Party Dialogue Discourse Parsing via Domain Integration](https://mlsanity.com/paper/2110.04526/) (2021)

## Built on by (most cited first, 24 of 24)

- [Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMs](https://mlsanity.com/paper/2402.03927/) (2024)
- [Exploring the Potential of Large Language Models in Computational Argumentation](https://mlsanity.com/paper/2311.09022/) (2023)
- [PlatoLM: Teaching LLMs in Multi-Round Dialogue via a User Simulator](https://mlsanity.com/paper/2308.11534/) (2023)
- [A Survey of the Evolution of Language Model-Based Dialogue Systems: Data, Task and Models](https://mlsanity.com/paper/2311.16789/) (2023)
- [Guided Profile Generation Improves Personalization with LLMs](https://mlsanity.com/paper/2409.13093/) (2024)
- [User Modeling in the Era of Large Language Models: Current Research and Future Directions](https://mlsanity.com/paper/2312.11518/) (2023)
- [Machine-assisted quantitizing designs: augmenting humanities and social sciences with artificial intelligence](https://mlsanity.com/paper/2309.14379/) (2023)
- [Improving Long Document Topic Segmentation Models With Enhanced Coherence Modeling](https://mlsanity.com/paper/2310.11772/) (2023)
- [Recent Trends in Linear Text Segmentation: a Survey](https://mlsanity.com/paper/2411.16613/) (2024)
- [Chunk Twice, Embed Once: A Systematic Study of Segmentation and Representation Trade-offs in Chemistry-Aware Retrieval-Augmented Generation](https://mlsanity.com/paper/2506.17277/) (2025)
- [Llamipa: An Incremental Discourse Parser](https://mlsanity.com/paper/2406.18256/) (2024)
- [Towards Multi-Level Transcript Segmentation: LoRA Fine-Tuning for Table-of-Contents Generation](https://mlsanity.com/paper/2601.02128/) (2026)
- [Advancing Topic Segmentation and Outline Generation in Chinese Texts: The Paragraph-level Topic Representation, Corpus, and Benchmark](https://mlsanity.com/paper/2305.14790/) (2023)
- [Unsupervised Mutual Learning of Discourse Parsing and Topic Segmentation in Dialogue](https://mlsanity.com/paper/2405.19799/) (2024)
- [Reading Between the Waves: Robust Topic Segmentation Using Inter-Sentence Audio Features](https://mlsanity.com/paper/2602.06647/) (2026)
- [Linear Semantic Segmentation for Low-Resource Spoken Dialects](https://mlsanity.com/paper/2605.06276/) (2026)
- [DiscoSum: Discourse-aware News Summarization](https://mlsanity.com/paper/2506.06930/) (2025)
- [DASH: Dialogue-Aware Similarity and Handshake Recognition for Topic Segmentation in Public-Channel Conversations](https://mlsanity.com/paper/2512.15042/) (2025)
- [CATCH: A Controllable Theme Detection Framework with Contextualized Clustering and Hierarchical Generation](https://mlsanity.com/paper/2512.21715/) (2025)
- [Codebook-Injected Dialogue Segmentation for Multi-Utterance Constructs Annotation: LLM-Assisted and Gold-Label-Free Evaluation](https://mlsanity.com/paper/2601.12061/) (2026)
- [From Documents to Segments: A Contextual Reformulation for Topic Assignment](https://mlsanity.com/paper/2605.17714/) (2026)
- [CobSeg: Coherence Boundary Modeling for Dialogue Topic Segmentation](https://mlsanity.com/paper/2605.30668/) (2026)
- [DraDDP: A Multimodal Multi-Party Dialogue Discourse Parsing Dataset](https://mlsanity.com/paper/2606.00012/) (2026)
- [Zero-Shot Cue-Grounded Topic Segmentation of Spoken Documents](https://mlsanity.com/paper/2609.34425/) (2026)

## Abstract (arXiv)

Large language models, like ChatGPT, have shown remarkable capability in many downstream tasks, yet their ability to understand discourse structures of dialogues remains less explored, where it requires higher level capabilities of understanding and reasoning. In this paper, we aim to systematically inspect ChatGPT's performance in two discourse analysis tasks: topic segmentation and discourse parsing, focusing on its deep semantic understanding of linear and hierarchical discourse structures underlying dialogue. To instruct ChatGPT to complete these tasks, we initially craft a prompt template consisting of the task description, output format, and structured input. Then, we conduct experiments on four popular topic segmentation datasets and two discourse parsing datasets. The experimental results showcase that ChatGPT demonstrates proficiency in identifying topic structures in general-domain conversations yet struggles considerably in specific-domain conversations. We also found that ChatGPT hardly understands rhetorical structures that are more complex than topic structures. Our deeper investigation indicates that ChatGPT can give more reasonable topic structures than human annotations but only linearly parses the hierarchical rhetorical structures. In addition, we delve into the impact of in-context learning (e.g., chain-of-thought) on ChatGPT and conduct the ablation study on various prompt components, which can provide a research foundation for future work. The code is available at urlhttps://github.com/yxfanSuda/GPTforDDA.