mlsanityData to 7 Oct 2026

Unraveling ChatGPT: A Critical Analysis of AI-Generated Goal-Oriented Dialogues and Annotations

arXiv 2305.14556 ↗First posted 23 May 2023cs.CL cs.AIcited by 14

Not on the map. Why?

Abstract (arXiv)

Large pre-trained language models have exhibited unprecedented capabilities in producing high-quality text via prompting techniques. This fact introduces new possibilities for data collection and annotation, particularly in situations where such data is scarce, complex to gather, expensive, or even sensitive. In this paper, we explore the potential of these models to generate and annotate goal-oriented dialogues, and conduct an in-depth analysis to evaluate their quality. Our experiments employ ChatGPT, and encompass three categories of goal-oriented dialogues (task-oriented, collaborative, and explanatory), two generation modes (interactive and one-shot), and two languages (English and Italian). Based on extensive human-based evaluations, we demonstrate that the quality of generated dialogues and annotations is on par with those generated by humans.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 17 of its 67 references that are in the map. First posted on arXiv 23 May 2023.

Builds on

17 of 67 references.

References · 17

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  7. 2017Deep reinforcement learning from human preferencescited by 3,846
  8. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  9. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  10. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  11. 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 760
  12. 2022Is GPT-3 a Good Data Annotator?cited by 208
5 more
  1. 2023Is ChatGPT better than Human Annotators? Potential and Limitations of ChatGPT in Explaining Implicit Hate Speechcited by 165
  2. 2021MultiWOZ 2.4: A Multi-Domain Task-Oriented Dialogue Dataset with Essential Annotation Corrections to Improve State Tracking Evaluationcited by 86
  3. 2020Recent Neural Methods on Slot Filling and Intent Classification for Task-Oriented Dialogue Systems: A Surveycited by 51
  4. 2022Towards Fair Evaluation of Dialogue State Tracking by Flexible Incorporation of Turn-level Performancescited by 13
  5. 2022"Mama Always Had a Way of Explaining Things So I Could Understand'': A Dialogue Corpus for Learning to Construct Explanationscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 4; 2025: 5; 2026: 2.

14 papers.

Sort:

  1. 2024Data Augmentation using Large Language Models: Data Perspectives, Learning Paradigms and Challengescited by 30
  2. 2024GPTs Are Multilingual Annotators for Sequence Generation Taskscited by 11
  3. 2023Conversational Feedback in Scripted versus Spontaneous Dialogues: A Comparative Analysiscited by 10
  4. 2023GPT Self-Supervision for a Better Data Annotatorcited by 5
  5. 2024Stephanie: Step-by-Step Dialogues for Mimicking Human Interactions in Social Conversationscited by 5
  6. 2023TOD-Flow: Modeling the Structure of Task-Oriented Dialoguescited by 4
  7. 2025An evaluation of LLMs and Google Translate for translation of selected Indian languages via sentiment and semantic analysescited by 2
  8. 2025SPADE: Structured Prompting Augmentation for Dialogue Enhancement in Machine-Generated Text Detectioncited by 1
  9. 2025Positional Bias in Binary Question Answering: How Uncertainty Shapes Model Preferencescited by 1
  10. 2024Training Zero-Shot Generalizable End-to-End Task-Oriented Dialog System Without Turn-level Dialog Annotationscited by 0
  11. 2025Improving Multi-turn Task Completion in Task-Oriented Dialog Systems via Prompt Chaining and Fine-Grained Feedbackcited by 0
  12. 2025Text Production and Comprehension by Human and Artificial Intelligence: Interdisciplinary Workshop Reportcited by 0
2 more
  1. 2026Codebook-Injected Dialogue Segmentation for Multi-Utterance Constructs Annotation: LLM-Assisted and Gold-Label-Free Evaluationcited by 0
  2. 2026Domain-Adapted Retrieval for In-Context Annotation of Pedagogical Dialogue Actscited by 0