mlsanityData to 14 Oct 2026

Generalizable Chain-of-Thought Prompting in Mixed-task Scenarios with Large Language Models

arXiv 2310.06692 ↗First posted 10 Oct 2023cs.CL cs.AIcited by 11

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Large language models (LLMs) have unveiled remarkable reasoning capabilities by exploiting chain-of-thought (CoT) prompting, which generates intermediate reasoning chains to serve as the rationale for deriving the answer. However, current CoT methods either simply employ general prompts such as Let's think step by step, or heavily rely on pre-defined task-specific demonstrations to attain preferable performances, thereby engendering an inescapable gap between performance and generalization. To bridge this gap, we propose GeM-CoT, a Generalizable CoT prompting mechanism in Mixed-task scenarios where the type of input questions is unknown. GeM-CoT first categorizes the question type and subsequently samples or constructs demonstrations from the corresponding data pool in an automatic pattern. With this technical design, GeM-CoT simultaneously enjoys superior generalization capabilities and remarkable performances on 10 public reasoning tasks and 23 BBH tasks.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 48 of its 54 references that are in the map. First posted on arXiv 10 Oct 2023.

Builds on

48 of 54 references.

Closest ancestors · 13

Same direction, newest first

  1. 2023Chain-Of-Thought Prompting Under Streaming Batch: A Case Study1 Jun
  2. 2023Better Zero-Shot Reasoning with Self-Adaptive Prompting23 May
  3. 2023Active Prompting with Chain-of-Thought for Large Language Models23 Feb
  4. 2023Faithful Chain-of-Thought Reasoning31 Jan
  5. 2022Large Language Models Are Reasoning Teachers20 Dec
  6. 2022Large Language Models are Better Reasoners with Self-Verification19 Dec
  7. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them17 Oct
  8. 2022Automatic Chain of Thought Prompting in Large Language Models7 Oct
  9. 2022Text and Patterns: For Effective Chain of Thought, It Takes Two to Tango16 Sep
  10. 2022Large Language Models are Zero-Shot Reasoners24 May
  11. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Models21 May
  12. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Models21 Mar
1 more
  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 35

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  7. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  8. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  9. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  10. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
  11. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  12. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
23 more
  1. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,195
  2. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,075
  3. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
  4. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 878
  5. 2023Large Language Models as Optimizerscited by 864
  6. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
  7. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 650
  8. 2023LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Compositioncited by 364
  9. 2016Solving General Arithmetic Word Problemscited by 360
  10. 2019Multi-Task Deep Neural Networks for Natural Language Understandingcited by 350
  11. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoningcited by 334
  12. 2021A Diverse Corpus for Evaluating and Developing English Math Word Problem Solverscited by 318
  13. 2022Making Large Language Models Better Reasoners with Step-Aware Verifiercited by 233
  14. 2021ExT5: Towards Extreme Multi-Task Scaling for Transfer Learningcited by 161
  15. 2022Ask Me Anything: A simple strategy for prompting language modelscited by 155
  16. 2022Rationale-Augmented Ensembles in Language Modelscited by 111
  17. 2022CommonsenseQA 2.0: Exposing the Limits of AI through Gamificationcited by 109
  18. 2021CREAK: A Dataset for Commonsense Reasoning over Entity Knowledgecited by 88
  19. 2023Answering Questions by Meta-Reasoning over Multiple Chains of Thoughtcited by 87
  20. 2021Dict-BERT: Enhancing Language Model Pre-training with Dictionarycited by 38
  21. 2021Structural Pre-training for Dialogue Comprehensioncited by 20
  22. 2022Task Compass: Scaling Multi-task Pre-training with Task Prefixcited by 14
  23. 2023Meta-Reasoning: Semantics-Symbol Deconstruction for Large Language Modelscited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 5; 2025: 4; 2026: 1.

11 papers.

  1. 2025Benchmarking Reasoning Robustness in Large Language Modelscited by 30
  2. 2023Just Ask One More Time! Self-Agreement Improves Reasoning of Language Models in (Almost) All Scenarioscited by 18
  3. 2024Can We Verify Step by Step for Incorrect Answer Detection?cited by 15
  4. 2025Reasoning with OmniThought: A Large CoT Dataset with Verbosity and Cognitive Difficulty Annotationscited by 14
  5. 2024DCR: Divide-and-Conquer Reasoning for Multi-choice Question Answering with LLMscited by 5
  6. 2024Let's Be Self-generated via Step by Step: A Curriculum Learning Approach to Automated Reasoning with Large Language Modelscited by 4
  7. 2025The Imitation Game: Turing Machine Imitator is Length Generalizable Reasonercited by 3
  8. 2024CoT-Driven Framework for Short Text Classification: Enhancing and Transferring Capabilities from Large to Smaller Modelcited by 2
  9. 2025Adaptive and Resource-efficient Agentic AI Systems for Mobile and Embedded Devices: A Surveycited by 2
  10. 2024uTeBC-NLP at SemEval-2024 Task 9: Can LLMs be Lateral Thinkers?cited by 1
  11. 2026Breaking the Reward Barrier: Accelerating Tree-of-Thought Reasoning via Speculative Explorationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Enhancing Chain of Thought Prompting in Large Language Models via Reasoning Patterns
  2. 2022Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters
  3. 2023An automatically discovered chain-of-thought prompt generalizes to novel models and datasets
  4. 2024ChainLM: Empowering Large Language Models with Improved Chain-of-Thought Prompting
  5. 2024Training Nonlinear Transformers for Chain-of-Thought Inference: A Theoretical Generalization Analysis
  6. 2024Unveiling the Statistical Foundations of Chain-of-Thought Prompting Methods
  7. 2023Stress Testing Chain-of-Thought Prompting for Large Language Models
  8. 2025CoT Vectors: Transferring and Probing the Reasoning Mechanisms of LLMs