mlsanityData to 7 Oct 2026

Can Small Language Models Help Large Language Models Reason Better?: LM-Guided Chain-of-Thought

arXiv 2404.03414 ↗First posted 4 Apr 2024cs.CL cs.AIcited by 12

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

We introduce a novel framework, LM-Guided CoT, that leverages a lightweight (i.e., <1B) language model (LM) for guiding a black-box large (i.e., >10B) LM in reasoning tasks. Specifically, the lightweight LM first generates a rationale for each input instance. The Frozen large LM is then prompted to predict a task output based on the rationale generated by the lightweight LM. Our approach is resource-efficient in the sense that it only requires training the lightweight LM. We optimize the model through 1) knowledge distillation and 2) reinforcement learning from rationale-oriented and task-oriented reward signals. We assess our method with multi-hop extractive question answering (QA) benchmarks, HotpotQA, and 2WikiMultiHopQA. Experimental results show that our approach outperforms all baselines regarding answer prediction accuracy. We also find that reinforcement learning helps the model to produce higher-quality rationales with improved QA performance.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 17 of its 26 references that are in the map. First posted on arXiv 4 Apr 2024.

Builds on

17 of 26 references.

References · 17

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  3. 2022Emergent Abilities of Large Language Modelscited by 2,483
  4. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  5. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  6. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  7. 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 668
  8. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 554
  9. 2023Is ChatGPT a Good NLG Evaluator? A Preliminary Studycited by 420
  10. 2023GPTScore: Evaluate as You Desirecited by 357
  11. 2022Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matterscited by 221
  12. 2022Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanationscited by 159
5 more
  1. 2023Symbolic Chain-of-Thought Distillation: Small Models Can Also "Think" Step-by-Stepcited by 144
  2. 2023Verify-and-Edit: A Knowledge-Enhanced Chain-of-Thought Frameworkcited by 139
  3. 2023ReCEval: Evaluating Reasoning Chains via Correctness and Informativenesscited by 76
  4. 2023Are Machine Rationales (Not) Useful to Humans? Measuring and Improving Human Utility of Free-Text Rationalescited by 37
  5. 2023Sci-CoT: Leveraging Large Language Models for Enhanced Knowledge Distillation in Small Models for Scientific QAcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 8; 2026: 1.

12 papers.

  1. 2024InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationalescited by 100
  2. 2024Reinforcement Learning Enhanced LLMs: A Surveycited by 69
  3. 2024What is the Role of Small Models in the LLM Era: A Surveycited by 41
  4. 2025Communicating Activations Between Language Model Agentscited by 15
  5. 2025A Survey on Collaborating Small and Large Language Models for Performance, Cost-effectiveness, Cloud-edge Privacy, and Trustworthinesscited by 5
  6. 2025Language Models, Graph Searching, and Supervision Adulteration: When More Supervision is Less and How to Make More Morecited by 3
  7. 2025Menta: A Small Language Model for On-Device Mental Health Predictioncited by 3
  8. 2025Small Language Models (SLMs) Can Still Pack a Punch: A survey (updated 2026)cited by 2
  9. 2025When LLMs Team Up: The Emergence of Collaborative Affective Computingcited by 2
  10. 2025Are LLMs Enough for Hyperpartisan, Fake, Polarized and Harmful Content Detection? Evaluating In-Context Learning vs. Fine-Tuningcited by 1
  11. 2026It Takes 8 Tokens: Weak-to-Strong Off-Policy RL via Auxiliary Branchescited by 1
  12. 2025SmallPlan: Leverage Small Language Models for Sequential Path Planning with Simulation-Powered, LLM-Guided Distillationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Training Language Models to Reason Efficiently
  2. 2026The Art of Efficient Reasoning: Data, Reward, and Optimization
  3. 2025Code-enabled language models can outperform reasoning models on diverse tasks
  4. 2024Optimizing Language Model's Reasoning Abilities with Weak Supervision
  5. 2024Patience Is The Key to Large Language Model Reasoning
  6. 2025RAVR: Reference-Answer-guided Variational Reasoning for Large Language Models
  7. 2025LoongRL: Reinforcement Learning for Advanced Reasoning over Long Contexts
  8. 2025Thinker: Learning to Think Fast and Slow