mlsanityData to 14 Oct 2026

ChatGPT Incorrectness Detection in Software Reviews

arXiv 2403.16347 ↗First posted 25 Mar 2024cs.SE cs.AI cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

We conducted a survey of 135 software engineering (SE) practitioners to understand how they use Generative AI-based chatbots like ChatGPT for SE tasks. We find that they want to use ChatGPT for SE tasks like software library selection but often worry about the truthfulness of ChatGPT responses. We developed a suite of techniques and a tool called CID (ChatGPT Incorrectness Detector) to automatically test and detect the incorrectness in ChatGPT responses. CID is based on the iterative prompting to ChatGPT by asking it contextually similar but textually divergent questions (using an approach that utilizes metamorphic relationships in texts). The underlying principle in CID is that for a given question, a response that is different from other responses (across multiple incarnations of the question) is likely an incorrect response. In a benchmark study of library selection, we show that CID can detect incorrect responses from ChatGPT with an F1-score of 0.74 - 0.75.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 41 of its 78 references that are in the map. First posted on arXiv 25 Mar 2024.

Builds on

41 of 78 references.

References · 41

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  4. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
  5. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
  6. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,310
  7. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 1,006
  8. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
  9. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 906
  10. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 894
  11. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 783
  12. 2022Teaching Models to Express Their Uncertainty in Wordscited by 751
29 more
  1. 2021Deduplicating Training Data Makes Language Models Bettercited by 714
  2. 2023The Internal State of an LLM Knows When It's Lyingcited by 659
  3. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 571
  4. 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 445
  5. 2021Unified Pre-training for Program Understanding and Generationcited by 366
  6. 2021Measuring and Improving Consistency in Pretrained Language Modelscited by 364
  7. 2021Entity-Based Knowledge Conflicts in Question Answeringcited by 356
  8. 2023How Language Model Hallucinations Can Snowballcited by 311
  9. 2020Reducing conversational agents' overconfidence through linguistic calibrationcited by 210
  10. 2023A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generationcited by 206
  11. 2022Factuality Enhanced Language Models for Open-Ended Text Generationcited by 202
  12. 2022Fine-tuning language models to find agreement among humans with diverse preferencescited by 189
  13. 2022Few-shot training LLMs for project-specific code-summarizationcited by 129
  14. 2023Keep the Conversation Going: Fixing 162 out of 337 bugs for 0.42 each using ChatGPTcited by 125
  15. 2019Handling Divergent Reference Texts when Evaluating Table-to-Text Generationcited by 119
  16. 2023Is ChatGPT the Ultimate Programming Assistant -- How far is it?cited by 76
  17. 2022Plausible May Not Be Faithful: Probing Object Hallucination in Vision-Language Pre-trainingcited by 72
  18. 2021CoTexT: Multi-task Learning with Code-Text Transformercited by 61
  19. 2022Automatic Code Documentation Generation Using GPT-3cited by 37
  20. 2020From Hero to Z'eroe: A Benchmark of Low-Level Adversarial Attackscited by 36
  21. 2021Focus Attention: Promoting Faithfulness and Diversity in Summarizationcited by 30
  22. 2020Revisiting Challenges in Data-to-Text Generation with Fact Groundingcited by 27
  23. 2021Automatic Program Repair with OpenAI's Codex: Evaluating QuixBugscited by 25
  24. 2020Machine Reading Comprehension: The Role of Contextualized Language Models and Beyondcited by 21
  25. 2019Modeling Fluency and Faithfulness for Diverse Neural Machine Translationcited by 13
  26. 2020Selecting third-party libraries: The practitioners' perspectivecited by 9
  27. 2021Understanding How and Why Developers Seek and Analyze API-related Opinionscited by 4
  28. 2023Combining Contexts from Multiple Sources for Documentation-Specific Code Example Generationcited by 2
  29. 2021Exploring Factors and Metrics to Select Open Source Software Components for Integration: An Empirical Studycited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 1.

5 papers.

  1. 2025Reflections on the Reproducibility of Commercial LLM Performance in Empirical Software Engineering Studiescited by 12
  2. 2025Can We Enhance Bug Report Quality Using LLMs?: An Empirical Study of LLM-Based Bug Report Generationcited by 3
  3. 2025HFuzzer: Testing Large Language Models for Package Hallucinations via Phrase-based Fuzzingcited by 1
  4. 2025ArchISMiner: A Framework for Automatic Mining of Architectural Issue-Solution Pairs from Online Developer Communitiescited by 0
  5. 2026An Empirical Analysis of Static Analysis Methods for Detection and Mitigation of Code Library Hallucinationscited by 0