mlsanityData to 7 Oct 2026

Active Prompting with Chain-of-Thought for Large Language Models

arXiv 2302.12246 ↗First posted 23 Feb 2023cs.CLcited by 154

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

The increasing scale of large language models (LLMs) brings emergent abilities to various complex tasks requiring reasoning, such as arithmetic and commonsense reasoning. It is known that the effective design of task-specific prompts is critical for LLMs' ability to produce high-quality answers. In particular, an effective approach for complex question-and-answer tasks is example-based prompting with chain-of-thought (CoT) reasoning, which significantly improves the performance of LLMs. However, current CoT methods rely on a fixed set of human-annotated exemplars, which are not necessarily the most effective examples for different tasks. This paper proposes a new method, Active-Prompt, to adapt LLMs to different tasks with task-specific example prompts (annotated with human-designed CoT reasoning). For this purpose, we propose a solution to the key problem of determining which questions are the most important and helpful ones to annotate from a pool of task-specific queries. By borrowing ideas from the related problem of uncertainty-based active learning, we introduce several metrics to characterize the uncertainty so as to select the most uncertain questions for annotation. Experimental results demonstrate the superiority of our proposed method, achieving state-of-the-art on eight complex reasoning tasks. Further analyses of different uncertainty metrics, pool sizes, zero-shot learning, and accuracy-uncertainty relationship demonstrate the effectiveness of our method. Our code will be available at https://github.com/shizhediao/active-prompt.

Key facts. Cited by 154 papers in the map (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 66 of its 91 references that are in the map. First posted on arXiv 23 Feb 2023.

Builds on

66 of 91 references.

Closest ancestors · 6

Same direction, newest first

  1. 2022Automatic Chain of Thought Prompting in Large Language Models7 Oct
  2. 2022Complexity-Based Prompting for Multi-Step Reasoning3 Oct
  3. 2022Large Language Models are Zero-Shot Reasoners24 May
  4. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Models21 May
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Models21 Mar
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 60

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2017On Calibration of Modern Neural Networkscited by 4,860
  7. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  8. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  9. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  10. 2022Emergent Abilities of Large Language Modelscited by 2,483
  11. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  12. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
38 more
  1. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  2. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  3. 2019How Can We Know What Language Models Know?cited by 953
  4. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  5. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  6. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  7. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  8. 2021BARTScore: Evaluating Generated Text as Text Generationcited by 719
  9. 2021GPT Understands, Toocited by 715
  10. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  11. 2022Large Language Models Can Self-Improvecited by 656
  12. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 653
  13. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
  14. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 557
  15. 2020AdapterHub: A Framework for Adapting Transformerscited by 441
  16. 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 402
  17. 2021Learning How to Ask: Querying LMs with Mixtures of Soft Promptscited by 376
  18. 2016Solving General Arithmetic Word Problemscited by 353
  19. 2018SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inferencecited by 333
  20. 2021A Diverse Corpus for Evaluating and Developing English Math Word Problem Solverscited by 314
  21. 2019Abductive Commonsense Reasoningcited by 300
  22. 2022UL2: Unifying Language Learning Paradigmscited by 292
  23. 2021Factual Probing Is [MASK]: Learning vs. Learning to Recallcited by 291
  24. 2022Prompting GPT-3 To Be Reliablecited by 258
  25. 2022Selective Annotation Makes Language Models Better Few-Shot Learnerscited by 242
  26. 2021PTR: Prompt Tuning with Rules for Text Classificationcited by 238
  27. 2022Making Large Language Models Better Reasoners with Step-Aware Verifiercited by 231
  28. 2021WARP: Word-level Adversarial ReProgrammingcited by 220
  29. 2020ReClor: A Reading Comprehension Dataset Requiring Logical Reasoningcited by 211
  30. 2019Commonsense Knowledge Mining from Pretrained Modelscited by 189
  31. 2020Scalable Multi-Hop Relational Reasoning for Knowledge-Aware Question Answeringcited by 122
  32. 2022Compositional Semantic Parsing with Large Language Modelscited by 95
  33. 2021BERTese: Learning to Speak to BERTcited by 82
  34. 2020Logic-Guided Data Augmentation and Regularization for Consistent Question Answeringcited by 77
  35. 2022Black-box Prompt Learning for Pre-trained Language Modelscited by 70
  36. 2019Cognitive Graph for Multi-Hop Reading Comprehension at Scalecited by 65
  37. 2021Revisiting Uncertainty-based Query Strategies for Active Learning with Transformerscited by 59
  38. 2018Knowledgeable Reader: Enhancing Cloze-Style Reading Comprehension with External Commonsense Knowledgecited by 48

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 43; 2024: 49; 2025: 41; 2026: 21.

154 papers.

Sort:

By area:

  1. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
  2. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 751
  3. 2024A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applicationscited by 428
  4. 2024Mixture-of-Agents Enhances Large Language Model Capabilitiescited by 420
  5. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 352
  6. 2024Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researcherscited by 348
  7. 2023Better Zero-Shot Reasoning with Role-Play Promptingcited by 220
  8. 2023Progressive-Hint Prompting Improves Reasoning in Large Language Modelscited by 147
  9. 2023DetGPT: Detect What You Need via Reasoningcited by 112
  10. 2024LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuningcited by 88
  11. 2023Exchange-of-Thought: Enhancing Large Language Model Capabilities through Cross-Model Communicationcited by 78
  12. 2023In-Context Learning with Iterative Demonstration Selectioncited by 74
38 more
  1. 2024BadChain: Backdoor Chain-of-Thought Prompting for Large Language Modelscited by 74
  2. 2023Unleashing the potential of prompt engineering for large language modelscited by 72
  3. 2023Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agentscited by 72
  4. 2023Cue-CoT: Chain-of-thought Prompting for Responding to In-depth Dialogue Questions with LLMscited by 62
  5. 2025ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inferencecited by 57
  6. 2024Follow the Rules: Reasoning for Video Anomaly Detection with Large Language Modelscited by 55
  7. 2023Fairness-guided Few-shot Prompting for Large Language Modelscited by 53
  8. 2023Active Learning Principles for In-Context Learning with Large Language Modelscited by 53
  9. 2024Internal Consistency and Self-Feedback in Large Language Models: A Surveycited by 49
  10. 2024A Survey of Prompt Engineering Methods in Large Language Models for Different NLP Taskscited by 46
  11. 2025Think Twice: Enhancing LLM Reasoning by Scaling Multi-round Test-time Thinkingcited by 46
  12. 2023Large Language Models as Tax Attorneys: A Case Study in Legal Capabilities Emergencecited by 42
  13. 2023Forward-Backward Reasoning in Large Language Models for Mathematical Verificationcited by 39
  14. 2023Which Examples to Annotate for In-Context Learning? Towards Effective and Efficient Selectioncited by 36
  15. 2024Cantor: Inspiring Multimodal Chain-of-Thought of MLLMcited by 34
  16. 2023Interactive Natural Language Processingcited by 31
  17. 2024Exploring Autonomous Agents through the Lens of Large Language Models: A Reviewcited by 30
  18. 2023CodeCoT: Tackling Code Syntax Errors in CoT Reasoning for Code Generationcited by 29
  19. 2023Revisiting Large Language Models as Zero-shot Relation Extractorscited by 29
  20. 2023Let GPT be a Math Tutor: Teaching Math Word Problem Solvers with Customized Exercise Generationcited by 27
  21. 2024Dynamic and Adaptive Feature Generation with LLMcited by 27
  22. 2023IDEAL: Influence-Driven Selective Annotations Empower In-Context Learners in Large Language Modelscited by 26
  23. 2024Cofca: A Step-Wise Counterfactual Multi-hop QA benchmarkcited by 26
  24. 2024Instance-adaptive Zero-shot Chain-of-Thought Promptingcited by 25
  25. 2024On Overcoming Miscalibrated Conversational Priors in LLM-based Chatbotscited by 23
  26. 2024Boosting of Thoughts: Trial-and-Error Problem Solving with Large Language Modelscited by 22
  27. 2024Can LLMs Solve longer Math Word Problems Better?cited by 22
  28. 2024Keypoint-based Progressive Chain-of-Thought Distillation for LLMscited by 22
  29. 2023Language models are weak learnerscited by 21
  30. 2023Prompting Large Language Models with Chain-of-Thought for Few-Shot Knowledge Base Question Generationcited by 21
  31. 2023CohortGPT: An Enhanced GPT for Participant Recruitment in Clinical Studycited by 19
  32. 2024Code Less, Align More: Efficient LLM Fine-tuning for Code Generation with Data Pruningcited by 19
  33. 2023Just Ask One More Time! Self-Agreement Improves Reasoning of Language Models in (Almost) All Scenarioscited by 18
  34. 2024Should We Really Edit Language Models? On the Evaluation of Edited Language Modelscited by 18
  35. 2025OnePiece: Bringing Context Engineering and Reasoning to Industrial Cascade Ranking Systemcited by 18
  36. 2023Enhancing Chain-of-Thoughts Prompting with Iterative Bootstrapping in Large Language Modelscited by 16
  37. 2023An Empirical Study of Translation Hypothesis Ensembling with Large Language Modelscited by 16
  38. 2025Causal Sufficiency and Necessity Improves Chain-of-Thought Reasoningcited by 16

50 most cited of 154 All 154 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Enhancing Chain of Thought Prompting in Large Language Models via Reasoning Patterns
  2. 2023An automatically discovered chain-of-thought prompt generalizes to novel models and datasets
  3. 2024Unveiling the Statistical Foundations of Chain-of-Thought Prompting Methods
  4. 2024Think Beyond Size: Adaptive Prompting for More Effective Reasoning
  5. 2024ChainLM: Empowering Large Language Models with Improved Chain-of-Thought Prompting
  6. 2023Large Language Models as Analogical Reasoners
  7. 2023R3 Prompting: Review, Rephrase and Resolve for Chain-of-Thought Reasoning in Large Language Models under Noisy Context
  8. 2024Chain-of-Thought Reasoning Without Prompting