mlsanityData to 7 Oct 2026

Rule-Guided Feedback: Enhancing Reasoning by Enforcing Rule Adherence in Large Language Models

arXiv 2503.11336 ↗First posted 14 Mar 2025cs.CLcited by 3

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

In this paper, we introduce Rule-Guided Feedback (RGF), a framework designed to enhance Large Language Model (LLM) performance through structured rule adherence and strategic information seeking. RGF implements a teacher-student paradigm where rule-following is forced through established guidelines. Our framework employs a Teacher model that rigorously evaluates each student output against task-specific rules, providing constructive guidance rather than direct answers when detecting deviations. This iterative feedback loop serves two crucial purposes: maintaining solutions within defined constraints and encouraging proactive information seeking to resolve uncertainties. We evaluate RGF on diverse tasks including Checkmate-in-One puzzles, Sonnet Writing, Penguins-In-a-Table classification, GSM8k, and StrategyQA. Our findings suggest that structured feedback mechanisms can significantly enhance LLMs' performance across various domains.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 14 of its 26 references that are in the map. First posted on arXiv 14 Mar 2025.

Builds on

14 of 26 references.

References · 14

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  5. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  6. 2022Large Language Models Are Reasoning Teacherscited by 318
  7. 2022Teaching Small Language Models to Reasoncited by 279
  8. 2022Distilling Reasoning Capabilities into Smaller Language Modelscited by 155
  9. 2023ExpertPrompting: Instructing Large Language Models to be Distinguished Expertscited by 140
  10. 2024Meta-Prompting: Enhancing Language Models with Task-Agnostic Scaffoldingcited by 115
  11. 2023Large Language Models can Learn Rulescited by 66
  12. 2024SoFA: Shielded On-the-fly Alignment via Priority Rule Followingcited by 18
2 more
  1. 2024ASGEA: Exploiting Logic Rules from Align-Subgraphs for Entity Alignmentcited by 5
  2. 2023Characterizing tradeoffs between teaching via language and demonstrations in multi-agent systemscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025RuleReasoner: Reinforced Rule-based Reasoning via Domain-aware Dynamic Samplingcited by 6
  2. 2025Sensory-Motor Control with Large Language Models via Iterative Policy Refinementcited by 0
  3. 2026Deep Interaction: An Efficient Human-AI Interaction Method for Large Reasoning Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024IDEA: Enhancing the Rule Learning Ability of Large Language Model Agent through Induction, Deduction, and Abduction
  2. 2023FollowEval: A Multi-Dimensional Benchmark for Assessing the Instruction-Following Capability of Large Language Models
  3. 2024TPD: Enhancing Student Language Model Reasoning via Principle Discovery and Guidance
  4. 2025THiNK: Can Large Language Models Think-aloud?
  5. 2026Beyond Factual Knowledge: Benchmarking and Learning Step-Level Procedural Rule Reasoning in Large Language Models
  6. 2023MAF: Multi-Aspect Feedback for Improving Reasoning in Large Language Models
  7. 2025Attentive Reasoning Queries: A Systematic Method for Optimizing Instruction-Following in Large Language Models
  8. 2024A NotSo Simple Way to Beat Simple Bench