mlsanityData to 7 Oct 2026

Verify-and-Edit: A Knowledge-Enhanced Chain-of-Thought Framework

arXiv 2305.03268 ↗First posted 5 May 2023cs.CLcited by 139

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

As large language models (LLMs) have become the norm in NLP, demonstrating good performance in generation and reasoning tasks, one of its most fatal disadvantages is the lack of factual correctness. Generating unfactual texts not only leads to lower performances but also degrades the trust and validity of their applications. Chain-of-Thought (CoT) prompting improves trust and model performance on complex reasoning tasks by generating interpretable reasoning chains, but still suffers from factuality concerns in knowledge-intensive tasks. In this paper, we propose the Verify-and-Edit framework for CoT prompting, which seeks to increase prediction factuality by post-editing reasoning chains according to external knowledge. Building on top of GPT-3, our framework lead to accuracy improvements in multiple open-domain question-answering tasks.

Key facts. Cited by 139 papers in the map (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 21 of its 35 references that are in the map. First posted on arXiv 5 May 2023.

Builds on

21 of 35 references.

Closest ancestors · 3

Same direction, newest first

  1. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Models21 May
  2. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Models21 Mar
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 18

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  7. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  8. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  9. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  10. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  11. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
  12. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
6 more
  1. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  2. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoningcited by 331
  3. 2022ROSCOE: A Suite of Metrics for Scoring Step-by-Step Reasoningcited by 175
  4. 2022The Unreliability of Explanations in Few-shot Prompting for Textual Reasoningcited by 169
  5. 2021Reframing Human-AI Collaboration for Generating Free-Text Explanationscited by 125
  6. 2023Can ChatGPT-like Generative Models Guarantee Factual Accuracy? On the Mistakes of New Generation Search Enginescited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 26; 2024: 67; 2025: 35; 2026: 11.

139 papers.

Sort:

By area:

  1. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  2. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  3. 2024Hallucination is Inevitable: An Innate Limitation of Large Language Modelscited by 329
  4. 2023Chain-of-Verification Reduces Hallucination in Large Language Modelscited by 317
  5. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 230
  6. 2023Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Futurecited by 184
  7. 2023UniChart: A Universal Vision-language Pretrained Model for Chart Comprehension and Reasoningcited by 158
  8. 2023Chain-of-Knowledge: Grounding Large Language Models via Dynamic Knowledge Adapting over Heterogeneous Sourcescited by 121
  9. 2024Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoningcited by 110
  10. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 108
  11. 2024InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationalescited by 100
  12. 2024Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMscited by 94
38 more
  1. 2024The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Modelscited by 89
  2. 2023Cognitive Mirage: A Review of Hallucinations in Large Language Modelscited by 83
  3. 2023In-Context Learning with Iterative Demonstration Selectioncited by 74
  4. 2023Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agentscited by 72
  5. 2023Making Language Models Better Tool Learners with Execution Feedbackcited by 69
  6. 2023WikiChat: Stopping the Hallucination of Large Language Model Chatbots by Few-Shot Grounding on Wikipediacited by 67
  7. 2024Rejection Improves Reliability: Training LLMs to Refuse Unknown Questions Using RL from Knowledge Feedbackcited by 64
  8. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 63
  9. 2024Building Guardrails for Large Language Modelscited by 61
  10. 2024Agents in Software Engineering: Survey, Landscape, and Visioncited by 60
  11. 2024Generate-then-Ground in Retrieval-Augmented Generation for Multi-hop Question Answeringcited by 51
  12. 2024Beyond Chain-of-Thought: A Survey of Chain-of-X Paradigms for LLMscited by 48
  13. 2024Efficient Prompting Methods for Large Language Models: A Surveycited by 46
  14. 2024A Survey of Prompt Engineering Methods in Large Language Models for Different NLP Taskscited by 46
  15. 2024Paths-over-Graph: Knowledge Graph Empowered Large Language Model Reasoningcited by 44
  16. 2024Think Twice Before Trusting: Self-Detection for Large Language Models through Comprehensive Answer Reflectioncited by 40
  17. 2023Ever: Mitigating Hallucination in Large Language Models through Real-Time Verification and Rectificationcited by 38
  18. 2024RAG-Star: Enhancing Deliberative Reasoning with Retrieval Augmented Verification and Refinementcited by 38
  19. 2023Towards Benchmarking and Improving the Temporal Reasoning Capability of Large Language Modelscited by 37
  20. 2024Reducing Tool Hallucination via Reliability Alignmentcited by 37
  21. 2023Halo: Estimation and Reduction of Hallucinations in Open-Source Weak Large Language Modelscited by 36
  22. 2024Knowledge Boundary of Large Language Models: A Surveycited by 36
  23. 2023How Do Large Language Models Capture the Ever-changing World Knowledge? A Review of Recent Advancescited by 33
  24. 2023Self-Checker: Plug-and-Play Modules for Fact-Checking with Large Language Modelscited by 31
  25. 2024Improving Uncertainty Quantification in Large Language Models via Semantic Embeddingscited by 30
  26. 2023Trends in Integration of Knowledge and Large Language Models: A Survey and Taxonomy of Methods, Benchmarks, and Applicationscited by 28
  27. 2024Rowen: Adaptive Retrieval-Augmented Generation for Hallucination Mitigation in LLMscited by 27
  28. 2023Mathematical Language Models: A Surveycited by 24
  29. 2024Citation-Enhanced Generation for LLM-based Chatbotscited by 24
  30. 2024Can We Further Elicit Reasoning in LLMs? Critic-Guided Planning with Retrieval-Augmentation for Solving Challenging Taskscited by 22
  31. 2024A Survey on the Honesty of Large Language Modelscited by 21
  32. 2024KnowPO: Knowledge-aware Preference Optimization for Controllable Knowledge Selection in Retrieval-Augmented Language Modelscited by 20
  33. 2024KGARevion: An AI Agent for Knowledge-Intensive Biomedical QAcited by 20
  34. 2024MINTQA: A Multi-Hop Question Answering Benchmark for Evaluating LLMs on New and Tail Knowledgecited by 20
  35. 2024Language Models Encode the Value of Numbers Linearlycited by 19
  36. 2024Retrieval-Augmented Generation with Estimation of Source Reliabilitycited by 19
  37. 2025Are Smarter LLMs Safer? Exploring Safety-Reasoning Trade-offs in Prompting and Fine-Tuningcited by 19
  38. 2023CoF-CoT: Enhancing Large Language Models with Coarse-to-Fine Chain-of-Thought Prompting for Multi-domain NLU Taskscited by 18

50 most cited of 139 All 139 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Boosting Language Models Reasoning with Chain-of-Knowledge Prompting
  2. 2024General Purpose Verification for Chain of Thought Prompting
  3. 2024On the Hardness of Faithful Chain-of-Thought Reasoning in Large Language Models
  4. 2022Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters
  5. 2025ECCoT: A Framework for Enhancing Effective Cognition via Chain of Thought in Large Language Model
  6. 2024Enhancing Chain of Thought Prompting in Large Language Models via Reasoning Patterns
  7. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models
  8. 2024A Chain-of-Thought Is as Strong as Its Weakest Link: A Benchmark for Verifiers of Reasoning Chains