mlsanityData to 14 Oct 2026

Enhancing Chain-of-Thoughts Prompting with Iterative Bootstrapping in Large Language Models

arXiv 2304.11657 ↗First posted 23 Apr 2023cs.CL cs.AIcited by 16

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

also in 1 more
Abstract (arXiv)

Large language models (LLMs) can achieve highly effective performance on various reasoning tasks by incorporating step-by-step chain-of-thought (CoT) prompting as demonstrations. However, the reasoning chains of demonstrations generated by LLMs are prone to errors, which can subsequently lead to incorrect reasoning during inference. Furthermore, inappropriate exemplars (overly simplistic or complex), can affect overall performance among varying levels of difficulty. We introduce Iter-CoT (Iterative bootstrapping in Chain-of-Thoughts Prompting), an iterative bootstrapping approach for selecting exemplars and generating reasoning chains. By utilizing iterative bootstrapping, our approach enables LLMs to autonomously rectify errors, resulting in more precise and comprehensive reasoning chains. Simultaneously, our approach selects challenging yet answerable questions accompanied by reasoning chains as exemplars with a moderate level of difficulty, which enhances the LLMs' generalizability across varying levels of difficulty. Experimental results indicate that Iter-CoT exhibits superiority, achieving competitive performance across three distinct reasoning tasks on ten datasets.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 35 of its 68 references that are in the map. First posted on arXiv 23 Apr 2023.

Builds on

35 of 68 references.

Closest ancestors · 9

Same direction, newest first

  1. 2023Automatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled Data24 Feb
  2. 2023Active Prompting with Chain-of-Thought for Large Language Models23 Feb
  3. 2023Synthetic Prompting: Generating Chain-of-Thought Demonstrations for Large Language Models1 Feb
  4. 2022Automatic Chain of Thought Prompting in Large Language Models7 Oct
  5. 2022Complexity-Based Prompting for Multi-Step Reasoning3 Oct
  6. 2022Large Language Models are Zero-Shot Reasoners24 May
  7. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Models21 May
  8. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Models21 Mar
  9. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 26

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  6. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  7. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  8. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
  9. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  10. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
  11. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,202
  12. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,075
14 more
  1. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 940
  2. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 878
  3. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
  4. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 662
  5. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 650
  6. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 559
  7. 2021A Diverse Corpus for Evaluating and Developing English Math Word Problem Solverscited by 318
  8. 2023ART: Automatic multi-step reasoning and tool-use for large language modelscited by 197
  9. 2021Poolingformer: Long Document Modeling with Pooling Attentioncited by 60
  10. 2022Reasoning Like Program Executorscited by 45
  11. 2019A Multi-Type Multi-Span Network for Reading Comprehension that Requires Discrete Reasoningcited by 41
  12. 2021Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attentioncited by 40
  13. 2021MWPToolkit: An Open-Source Framework for Deep Learning-Based Math Word Problem Solverscited by 39
  14. 2023Second Thoughts are Best: Learning to Re-Align With Human Values from Text Editscited by 27

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 9; 2025: 2; 2026: 2.

16 papers.

Sort:

By area:

  1. 2024Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Securitycited by 260
  2. 2023Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graphcited by 238
  3. 2024Unlocking the Capabilities of Thought: A Reasoning Boundary Framework to Quantify and Optimize Chain-of-Thoughtcited by 56
  4. 2024How Far Can Transformers Reason? The Globality Barrier and Inductive Scratchpadcited by 36
  5. 2023Pushing the Limits of ChatGPT on NLP Taskscited by 27
  6. 2024Improving Physics Reasoning in Large Language Models Using Mixture of Refinement Agentscited by 24
  7. 2024Mitigating Misleading Chain-of-Thought Reasoning with Selective Filteringcited by 10
  8. 2024Embodied CoT Distillation From LLM To Off-the-shelf Agentscited by 9
  9. 2023Self-Convinced Prompting: Few-Shot Question Answering with Repeated Introspectioncited by 7
  10. 2024DCR: Divide-and-Conquer Reasoning for Multi-choice Question Answering with LLMscited by 5
  11. 2024Multi-Turn Human-LLM Interaction Through the Lens of a Two-Way Intelligibility Protocolcited by 4
  12. 2025CountLoop: Training-Free High-Instance Image Generation via Iterative Agent Guidancecited by 4
4 more
  1. 2024FairCoT: Enhancing Fairness in Text-to-Image Generation via Chain of Thought Reasoning with Multimodal Large Language Modelscited by 3
  2. 2025Program of Thoughts for Financial Reasoning: Leveraging Dynamic In-Context Examples and Generative Retrievalcited by 2
  3. 2026Prompt Engineering for Scale Development in Generative Psychometricscited by 1
  4. 2026StraTune: Adaptive Selection of Revision Operators for Self-Evolving LLM Skillscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Enhancing Chain of Thought Prompting in Large Language Models via Reasoning Patterns
  2. 2024ChainLM: Empowering Large Language Models with Improved Chain-of-Thought Prompting
  3. 2025Early Stopping Chain-of-thoughts in Large Language Models
  4. 2022Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters
  5. 2023Boosting Language Models Reasoning with Chain-of-Knowledge Prompting
  6. 2024Fine-Tuning on Diverse Reasoning Chains Drives Within-Inference CoT Refinement in LLMs
  7. 2023An automatically discovered chain-of-thought prompt generalizes to novel models and datasets
  8. 2024Unveiling the Statistical Foundations of Chain-of-Thought Prompting Methods