mlsanityData to 7 Oct 2026

CoTEVer: Chain of Thought Prompting Annotation Toolkit for Explanation Verification

arXiv 2303.03628 ↗First posted 7 Mar 2023cs.CL cs.LGcited by 11

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Chain-of-thought (CoT) prompting enables large language models (LLMs) to solve complex reasoning tasks by generating an explanation before the final prediction. Despite it's promising ability, a critical downside of CoT prompting is that the performance is greatly affected by the factuality of the generated explanation. To improve the correctness of the explanations, fine-tuning language models with explanation data is needed. However, there exists only a few datasets that can be used for such approaches, and no data collection tool for building them. Thus, we introduce CoTEVer, a tool-kit for annotating the factual correctness of generated explanations and collecting revision data of wrong explanations. Furthermore, we suggest several use cases where the data collected with CoTEVer can be utilized for enhancing the faithfulness of explanations. Our toolkit is publicly available at https://github.com/SeungoneKim/CoTEVer.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 34 of its 49 references that are in the map. First posted on arXiv 7 Mar 2023.

Builds on

34 of 49 references.

Closest ancestors · 3

Same direction, newest first

  1. 2022Large Language Models are Zero-Shot Reasoners24 May
  2. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Models21 Mar
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 31

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  4. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  5. 2022Emergent Abilities of Large Language Modelscited by 2,483
  6. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  7. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  8. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  9. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  10. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  11. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895
  12. 2022Large Language Models Can Self-Improvecited by 656
19 more
  1. 2019Neural Text Generation with Unlikelihood Trainingcited by 489
  2. 2021Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Modelscited by 462
  3. 2021MetaICL: Learning to Learn In Contextcited by 450
  4. 2022Knowledge Unlearning for Mitigating Privacy Risks in Language Modelscited by 424
  5. 2022PromptSource: An Integrated Development Environment and Repository for Natural Language Promptscited by 288
  6. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 286
  7. 2021Symbolic Knowledge Distillation: from General Language Models to Commonsense Modelscited by 270
  8. 2022Making Large Language Models Better Reasoners with Step-Aware Verifiercited by 231
  9. 2022ROSCOE: A Suite of Metrics for Scoring Step-by-Step Reasoningcited by 175
  10. 2021Towards Continual Knowledge Learning of Language Modelscited by 172
  11. 2022DiffCSE: Difference-based Contrastive Learning for Sentence Embeddingscited by 131
  12. 2021Generate & Rank: A Multi-task Framework for Math Word Problemscited by 82
  13. 2023Exploring the Benefits of Training Expert Language Models over Instruction Tuningcited by 80
  14. 2021Memorisation versus Generalisation in Pre-trained Language Modelscited by 32
  15. 2022Guess the Instruction! Flipped Learning Makes Language Models Stronger Zero-Shot Learnerscited by 22
  16. 2022REV: Information-Theoretic Evaluation of Free-Text Rationalescited by 22
  17. 2022Dynatask: A Framework for Creating Dynamic AI Benchmark Taskscited by 16
  18. 2021AnnIE: An Annotation Platform for Constructing Complete Open Information Extraction Benchmarkcited by 7
  19. 2022The slurk Interaction Server Framework: Better Data for Better Dialog Modelscited by 7

Built on by

Cited by year

Citing papers per year: 2023: 4; 2024: 6; 2025: 1.

11 papers.

  1. 2023The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuningcited by 106
  2. 2023Knowledge-Driven CoT: Exploring Faithful Reasoning in LLMs for Knowledge-intensive Question Answeringcited by 97
  3. 2024The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Modelscited by 89
  4. 2023Towards Better Chain-of-Thought Prompting Strategies: A Surveycited by 62
  5. 2024Evaluating Language Models as Synthetic Data Generatorscited by 32
  6. 2024Self-Explore: Enhancing Mathematical Reasoning in Language Models with Fine-grained Rewardscited by 31
  7. 2024The "Colonial Impulse" of Natural Language Processing: An Audit of Bengali Sentiment Analysis Tools and Their Identity-based Biasescited by 16
  8. 2024Understanding Reasoning in Chain-of-Thought from the Hopfieldian Viewcited by 10
  9. 2025Ranking Free RAG: Replacing Re-ranking with Selection in RAG for Sensitive Domainscited by 5
  10. 2023Event-driven Real-time Retrieval in Web Searchcited by 1
  11. 2024ARTiST: Automated Text Simplification for Task Guidance in Augmented Realitycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024ChainLM: Empowering Large Language Models with Improved Chain-of-Thought Prompting
  2. 2022Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters
  3. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting
  4. 2023Boosting Language Models Reasoning with Chain-of-Knowledge Prompting
  5. 2024Unveiling the Statistical Foundations of Chain-of-Thought Prompting Methods
  6. 2023Faithful Chain-of-Thought Reasoning
  7. 2024Enhancing Chain of Thought Prompting in Large Language Models via Reasoning Patterns
  8. 2025Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens