mlsanityData to 14 Oct 2026

CoT-ICL Lab: A Synthetic Framework for Studying Chain-of-Thought Learning from In-Context Demonstrations

arXiv 2502.15132 ↗First posted 21 Feb 2025cs.CL cs.AI cs.LGcited by 4

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

We introduce CoT-ICL Lab, a framework and methodology to generate synthetic tokenized datasets and systematically study chain-of-thought (CoT) in-context learning (ICL) in language models. CoT-ICL Lab allows fine grained control over the complexity of in-context examples by decoupling (1) the causal structure involved in chain token generation from (2) the underlying token processing functions. We train decoder-only transformers (up to 700M parameters) on these datasets and show that CoT accelerates the accuracy transition to higher values across model sizes. In particular, we find that model depth is crucial for leveraging CoT with limited in-context examples, while more examples help shallow models match deeper model performance. Additionally, limiting the diversity of token processing functions throughout training improves causal structure learning via ICL. We also interpret these transitions by analyzing transformer embeddings and attention maps. Overall, CoT-ICL Lab serves as a simple yet powerful testbed for theoretical and empirical insights into ICL and CoT in language models.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 34 of its 45 references that are in the map. First posted on arXiv 21 Feb 2025.

Builds on

34 of 45 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024Deciphering the Factors Influencing the Efficacy of Chain-of-Thought: Probability, Memorization, and Noisy Reasoning1 Jul
  2. 2023Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Future27 Sep
  3. 2023Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective24 May
  4. 2023The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning23 May
  5. 2023Why think step by step? Reasoning emerges from the locality of experience7 Apr
  6. 2022Large Language Models are Zero-Shot Reasoners24 May
  7. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 27

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2024The Llama 3 Herd of Modelscited by 14,715
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  6. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  7. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  8. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 938
  9. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 886
  10. 2022Transformers learn in-context by gradient descentcited by 732
  11. 2022What Can Transformers Learn In-Context? A Case Study of Simple Function Classescited by 715
  12. 2023Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selectioncited by 298
15 more
  1. 2023Transformers learn to implement preconditioned gradient descent for in-context learningcited by 279
  2. 2023Transformers as Algorithms: Generalization and Stability in In-context Learningcited by 222
  3. 2024Liger Kernel: Efficient Triton Kernels for LLM Trainingcited by 130
  4. 2024The Evolution of Statistical Induction Heads: In-Context Learning Markov Chainscited by 118
  5. 2024How Transformers Learn Causal Structure with Gradient Descentcited by 118
  6. 2022Rationale-Augmented Ensembles in Language Modelscited by 111
  7. 2024In-Context Language Learning: Architectures and Algorithmscited by 89
  8. 2023In-Context Learning through the Bayesian Prismcited by 83
  9. 2023How Do Transformers Learn In-Context Beyond Simple Functions? A Case Study on Learning with Representationscited by 75
  10. 2023Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Modelscited by 50
  11. 2023The Mystery of In-Context Learning: A Comprehensive Survey on Interpretation and Analysiscited by 34
  12. 2024Pretrained transformer efficiently learns low-dimensional target functions in-contextcited by 31
  13. 2023In-context Learning and Gradient Descent Revisitedcited by 20
  14. 2025360Brew: A Decoder-only Foundation Model for Personalized Ranking and Recommendationcited by 12
  15. 2024DAG-aware Transformer for Causal Effect Estimationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2025TRUST: A Decentralized Framework for Auditing Large Language Model Reasoningcited by 1
  2. 2026Panning for Gold: Expanding Domain-Specific Knowledge Graphs with General Knowledgecited by 0
  3. 2026TRUST: A Framework for Decentralized AI Service v.0.1cited by 0
  4. 2026RT-J: Large-Scale Pretraining of Relational Transformers for Context-Efficient Predictionscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025The Curse of CoT: On the Limitations of Chain-of-Thought in In-Context Learning
  2. 2023Dissecting Chain-of-Thought: Compositionality through In-Context Filtering and Learning
  3. 2026ACIL: Auto Chain of Thoughts for In-Context Learning
  4. 2025To Think or Not to Think: The Hidden Cost of Meta-Training with Excessive CoT Examples
  5. 2025Illusion or Algorithm? Investigating Memorization, Emergence, and Symbolic Processing in In-Context Learning
  6. 2024Training Nonlinear Transformers for Chain-of-Thought Inference: A Theoretical Generalization Analysis
  7. 2024Expediting and Elevating Large Language Model Reasoning via Hidden Chain-of-Thought Decoding
  8. 2025SIM-CoT: Supervised Implicit Chain-of-Thought