mlsanityData to 14 Oct 2026

From Harm to Help: Turning Reasoning In-Context Demos into Assets for Reasoning LMs

arXiv 2509.23196 ↗First posted 27 Sep 2025cs.CLcited by 1

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Recent reasoning LLMs (RLMs), especially those trained with verifier-based reinforcement learning, often perform worse with few-shot CoT than with direct answering. We revisit this paradox using high-quality reasoning traces from DeepSeek-R1 as demonstrations and find that adding more exemplars consistently degrades accuracy, even when demonstrations are optimal. A detailed analysis reveals two mechanisms behind this decline: (i) semantic misguidance, where high textual similarity leads the model to treat the target as the same as the exemplar and to copy intermediate steps verbatim; and (ii) strategy transfer failure, where the model struggles to extract useful reasoning strategies and apply them to target questions. Guided by these, we introduce Insight-to-Solve (I2S), a sequential test-time procedure that turns demonstrations into explicit, reusable insights and derives a target-specific reasoning trace; optionally, the reasoning is self-refined for coherence and correctness (I2S+). Extensive experiments on diverse benchmarks show that I2S and I2S+ consistently outperform both direct answering and test-time scaling baselines across open- and closed-source models. Even for GPT models, our method helps: on AIME'25, GPT-4.1 rises by +14.0%, and o1-mini improves by +2.7% on AIME and +1.7% on GPQA, indicating that in-context demonstrations can be harnessed effectively via insight-refine-solve framework.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 46 of its 54 references that are in the map. First posted on arXiv 27 Sep 2025.

Builds on

46 of 54 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Reinforcing General Reasoning without Verifiers27 May
  2. 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar
  3. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan
  5. 2024AlphaMath Almost Zero: Process Supervision without Process6 May

Other references · 41

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2025Qwen3 Technical Reportcited by 8,820
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  7. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  8. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  9. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  10. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
  11. 2023Teaching Large Language Models to Self-Debugcited by 988
  12. 2025Group Sequence Policy Optimizationcited by 835
29 more
  1. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 783
  2. 2022Transformers learn in-context by gradient descentcited by 732
  3. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 708
  4. 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 564
  5. 2024Training Language Models to Self-Correct via Reinforcement Learningcited by 426
  6. 2024Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Modelscited by 290
  7. 2023Transformers learn to implement preconditioned gradient descent for in-context learningcited by 279
  8. 2023Query Rewriting for Retrieval-Augmented Large Language Modelscited by 213
  9. 2019Stochastic Beams and Where to Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacementcited by 211
  10. 2022Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizerscited by 204
  11. 2023Universal Self-Consistency for Large Language Model Generationcited by 203
  12. 2023Planning with Large Language Models for Code Generationcited by 182
  13. 2023Progressive-Hint Prompting Improves Reasoning in Large Language Modelscited by 147
  14. 2023Unified Demonstration Retriever for In-Context Learningcited by 142
  15. 2025Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scalingcited by 136
  16. 2024BoNBoN Alignment for Large Language Models and the Sweetness of Best-of-n Samplingcited by 121
  17. 2024Fast Best-of-N Decoding via Speculative Rejectioncited by 112
  18. 2024Small Language Models Need Strong Verifiers to Self-Correct Reasoningcited by 86
  19. 2023OVM, Outcome-supervised Value Models for Planning in Mathematical Reasoningcited by 85
  20. 2023In-Context Learning with Iterative Demonstration Selectioncited by 74
  21. 2025Revisiting the Test-Time Scaling of o1-like Models: Do they Truly Possess Test-Time Scaling Capabilities?cited by 70
  22. 2024TreeBoN: Enhancing Inference-Time Alignment with Speculative Tree-Search and Best-of-N Samplingcited by 61
  23. 2024Revisiting Demonstration Selection Strategies in In-Context Learningcited by 58
  24. 2024The Limits of Inference Scaling Through Resamplingcited by 53
  25. 2024When is Tree Search Useful for LLM Planning? It Depends on the Discriminatorcited by 47
  26. 2025The Curse of CoT: On the Limitations of Chain-of-Thought in In-Context Learningcited by 30
  27. 2025LLMs Can Generate a Better Answer by Aggregating Their Own Responsescited by 27
  28. 2025Revisiting Chain-of-Thought Prompting: Zero-shot Can Be Stronger than Few-shotcited by 12
  29. 2024AutoReason: Automatic Few-Shot Reasoning Decompositioncited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Lost in Adaptation: Layer-Selective Recovery of Temporal Reasoning in Video-Language Modelscited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2026Good Reasoning Makes Good Demonstrations: Implicit Reasoning Quality Supervision via In-Context Reinforcement Learning
  2. 2025RAVR: Reference-Answer-guided Variational Reasoning for Large Language Models
  3. 2026ExpRL: Exploratory RL for LLM Mid-Training
  4. 2025Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective
  5. 2025Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs
  6. 2026ALIVE: Awakening LLM Reasoning via Adversarial Learning and Instructive Verbal Evaluation
  7. 2026Training Reasoning Models on Saturated Problems via Failure-Prefix Conditioning
  8. 2026Beyond Reasoning: Reinforcement Learning Unlocks Parametric Knowledge in LLMs