mlsanityData to 7 Oct 2026

Problem-Solving Logic Guided Curriculum In-Context Learning for LLMs Complex Reasoning

arXiv 2502.15401 ↗First posted 21 Feb 2025cs.CL cs.AIcited by 12

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

In-context learning (ICL) can significantly enhance the complex reasoning capabilities of large language models (LLMs), with the key lying in the selection and ordering of demonstration examples. Previous methods typically relied on simple features to measure the relevance between examples. We argue that these features are not sufficient to reflect the intrinsic connections between examples. In this study, we propose a curriculum ICL strategy guided by problem-solving logic. We select demonstration examples by analyzing the problem-solving logic and order them based on curriculum learning. Specifically, we constructed a problem-solving logic instruction set based on the BREAK dataset and fine-tuned a language model to analyze the problem-solving logic of examples. Subsequently, we selected appropriate demonstration examples based on problem-solving logic and assessed their difficulty according to the number of problem-solving steps. In accordance with the principles of curriculum learning, we ordered the examples from easy to hard to serve as contextual prompts. Experimental results on multiple benchmarks indicate that our method outperforms previous ICL approaches in terms of performance and efficiency, effectively enhancing the complex reasoning capabilities of LLMs. Our project will be released at https://github.com/maxuetao/CurriculumICL

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 20 of its 68 references that are in the map. First posted on arXiv 21 Feb 2025.

Builds on

20 of 68 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Composition9 Oct
  2. 2023Cumulative Reasoning with Large Language Models8 Aug

Other references · 18

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  4. 2024Qwen2.5 Technical Reportcited by 4,713
  5. 2022A Survey on In-context Learningcited by 717
  6. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 684
  7. 2025LIMO: Less is More for Reasoningcited by 488
  8. 2023A Comprehensive Capability Analysis of GPT-3 and GPT-3.5 Series Modelscited by 244
  9. 2024In-Context Learning with Long-Context Models: An In-Depth Explorationcited by 126
  10. 2024In-context Learning with Retrieved Demonstrations for Language Models: A Surveycited by 69
  11. 2023Understanding In-Context Learning in Transformers and LLMs by Learning to Learn Discrete Functionscited by 62
  12. 2023In-context Example Selection with Influencescited by 60
6 more
  1. 2024Revisiting Demonstration Selection Strategies in In-Context Learningcited by 57
  2. 2024Let's Learn Step by Step: Enhancing In-Context Learning Ability with Curriculum Learningcited by 47
  3. 2024Strategic Data Ordering: Enhancing Large Language Model Performance through Curriculum Learningcited by 28
  4. 2024Dataset Decomposition: Faster LLM Training with Variable Sequence Length Curriculumcited by 25
  5. 2023Instruction Tuning with Human Curriculumcited by 15
  6. 2024DemoShapley: Valuation of Demonstrations for In-Context Learningcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 8; 2026: 4.

12 papers.

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  2. 2025Bottom-up Domain-specific Superintelligence: A Reliable Knowledge Graph is What We Needcited by 12
  3. 2025UORA: Uniform Orthogonal Reinitialization Adaptation in Parameter-Efficient Fine-Tuning of Large Modelscited by 11
  4. 2025The Role of Diversity in In-Context Learning for Large Language Modelscited by 6
  5. 2025Reasoning Steps as Curriculum: Using Depth of Thought as a Difficulty Signal for Tuning LLMscited by 3
  6. 2026MemGround: Long-Term Memory Evaluation Kit for Large Language Models in Gamified Scenarioscited by 2
  7. 2025SOLAR: Scalable Optimization of Large-scale Architecture for Reasoningcited by 1
  8. 2026MathMixup: Boosting LLM Mathematical Reasoning with Difficulty-Controllable Data Synthesis and Curriculum Learningcited by 1
  9. 2025SMART: Self-Generating and Self-Validating Multi-Dimensional Assessment for LLMs' Mathematical Problem Solvingcited by 0
  10. 2025STELAR-VISION: Self-Topology-Aware Efficient Learning for Aligned Reasoning in Visioncited by 0
  11. 2026SketchJudge: A Diagnostic Benchmark for Grading Hand-drawn Diagrams with Multimodal Large Language Modelscited by 0
  12. 2026Design-Specification Tiling for ICL-based CAD Code Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Multi-Step Reasoning with Large Language Models, a Survey
  2. 2024Benchmarking Large Language Models for Math Reasoning Tasks
  3. 2025Knowledge Augmented Complex Problem Solving with Large Language Models: A Survey
  4. 2022Teaching Algorithmic Reasoning via In-context Learning
  5. 2025Progressive Mastery: Customized Curriculum Learning with Guided Prompting for Mathematical Reasoning
  6. 2026Process In-Context Learning: Enhancing Mathematical Reasoning via Dynamic Demonstration Insertion
  7. 2023Unleashing the Creative Mind: Language Model As Hierarchical Policy For Improved Exploration on Challenging Problem Solving
  8. 2024Beyond Examples: High-level Automated Reasoning Paradigm in In-Context Learning via MCTS