mlsanityData to 7 Oct 2026

A Preliminary Evaluation of ChatGPT for Zero-shot Dialogue Understanding

arXiv 2304.04256 ↗First posted 9 Apr 2023cs.CLcited by 31

Not on the map. Why?

Abstract (arXiv)

Zero-shot dialogue understanding aims to enable dialogue to track the user's needs without any training data, which has gained increasing attention. In this work, we investigate the understanding ability of ChatGPT for zero-shot dialogue understanding tasks including spoken language understanding (SLU) and dialogue state tracking (DST). Experimental results on four popular benchmarks reveal the great potential of ChatGPT for zero-shot dialogue understanding. In addition, extensive analysis shows that ChatGPT benefits from the multi-turn interactive prompt in the DST task but struggles to perform slot filling for SLU. Finally, we summarize several unexpected behaviors of ChatGPT in dialogue understanding tasks, hoping to provide some insights for future research on building zero-shot dialogue understanding systems with Large Language Models (LLMs).

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Builds on 18 of its 26 references that are in the map. First posted on arXiv 9 Apr 2023.

Builds on

18 of 26 references.

References · 18

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  6. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  7. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 486
  8. 2018Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfacescited by 376
  9. 2023ChatIE: Zero-Shot Information Extraction via Chatting with ChatGPTcited by 250
  10. 2023Is ChatGPT A Good Translator? Yes With GPT-4 As The Enginecited by 172
  11. 2023Exploring the Limits of ChatGPT for Query or Aspect-based Text Summarizationcited by 122
  12. 2021MultiWOZ 2.4: A Multi-Domain Task-Oriented Dialogue Dataset with Essential Annotation Corrections to Improve State Tracking Evaluationcited by 86
6 more
  1. 2020CoSDA-ML: Multi-Lingual Code-Switching Data Augmentation for Zero-Shot Cross-Lingual NLPcited by 79
  2. 2020A Co-Interactive Transformer for Joint Slot Filling and Intent Detectioncited by 61
  3. 2021A Survey on Spoken Language Understanding: Recent Advances and New Frontierscited by 53
  4. 2022Description-Driven Task-Oriented Dialog Modelingcited by 50
  5. 2020Coach: A Coarse-to-Fine Approach for Cross-domain Slot Fillingcited by 38
  6. 2022"Do you follow me?": A Survey of Recent Approaches in Dialogue State Trackingcited by 22

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 9; 2024: 14; 2025: 5; 2026: 3.

31 papers.

Sort:

By area:

  1. 2024Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMscited by 224
  2. 2024Large Language Models Meet NLP: A Surveycited by 102
  3. 2023Cross-lingual Prompting: Improving Zero-shot Chain-of-Thought Reasoning across Languagescited by 96
  4. 2024Unlocking the Capabilities of Thought: A Reasoning Boundary Framework to Quantify and Optimize Chain-of-Thoughtcited by 56
  5. 2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontierscited by 49
  6. 2023Rethinking the Evaluation for Conversational Recommendation in the Era of Large Language Modelscited by 48
  7. 2024Evaluating Large Language Models as Generative User Simulators for Conversational Recommendationcited by 45
  8. 2023Exploring the Potential of Large Language Models in Computational Argumentationcited by 42
  9. 2023Multi-Task Instruction Tuning of LLaMa for Specific Scenarios: A Preliminary Study on Writing Assistancecited by 33
  10. 2024What Factors Affect Multi-Modal In-Context Learning? An In-Depth Explorationcited by 31
  11. 2023Large Language Models are Complex Table Parserscited by 28
  12. 2023Towards LLM-driven Dialogue State Trackingcited by 25
19 more
  1. 2024Can Small Language Models be Good Reasoners for Sequential Recommendation?cited by 25
  2. 2023Balancing Specialized and General Skills in LLMs: The Impact of Modern Tuning and Data Strategycited by 18
  3. 2023End-to-end Task-oriented Dialogue: A Survey of Tasks, Methods, and Future Directionscited by 17
  4. 2024Beyond the Known: Investigating LLMs Performance on Out-of-Domain Intent Detectioncited by 14
  5. 2025RBF++: Quantifying and Optimizing Reasoning Boundaries across Measurable and Unmeasurable Capabilities for Chain-of-Thought Reasoningcited by 4
  6. 2023Speech-based Slot Filling using Large Language Modelscited by 3
  7. 2024CroPrompt: Cross-task Interactive Prompting for Zero-shot Spoken Language Understandingcited by 3
  8. 2025LANID: LLM-assisted New Intent Discoverycited by 3
  9. 2025DMDTEval: An Evaluation and Analysis of LLMs on Disambiguation in Multi-domain Translationcited by 3
  10. 2024A Zero-Shot Open-Vocabulary Pipeline for Dialogue Understandingcited by 2
  11. 2025LLM-Assisted Automated Deductive Coding of Dialogue Data: Leveraging Dialogue-Specific Characteristics to Enhance Contextual Understandingcited by 2
  12. 2024ICE-GRT: Instruction Context Enhancement by Generative Reinforcement based Transformerscited by 1
  13. 2024Gradable ChatGPT Translation Evaluationcited by 1
  14. 2024Word Order's Impacts: Insights from Reordering and Generation Analysiscited by 1
  15. 2024Benchmarks Underestimate the Readiness of Multi-lingual Dialogue Agentscited by 1
  16. 2025Introducing Visual Scenes and Reasoning: A More Realistic Benchmark for Spoken Language Understandingcited by 1
  17. 2026Bridging Reasoning and Action: Hybrid LLM-RL Framework for Efficient Cross-Domain Task-Oriented Dialoguecited by 0
  18. 2026MC-PDD: Masked Corpus-Level Pretraining Data Detection for Black-Box Large Language Modelscited by 0
  19. 2026SFL-MTSC: Leveraging Semantic Frame-Level Multi-Task Self-Consistency for Robust Multi-Intent Spoken Language Understandingcited by 0