mlsanityData to 7 Oct 2026

RuleReasoner: Reinforced Rule-based Reasoning via Domain-aware Dynamic Sampling

arXiv 2506.08672 ↗First posted 10 Jun 2025cs.CL cs.AI cs.LGcited by 6

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Rule-based reasoning is acknowledged as one of the fundamental problems of reasoning. While recent studies show that large reasoning models (LRMs) have remarkable reasoning capabilities enhanced by reinforcement learning (RL), real applications still face severe challenges due to variations in rule formats, types, and complexity. To mitigate this issue, we introduce RuleReasoner, an effective method for rule-based reasoning via a wide collection of curated tasks and a novel domain-aware dynamic sampling approach in RL. Specifically, RuleReasoner resamples each training batch by updating the domain weights based on historical rewards. This facilitates domain balance and active learning schedules for RL, obviating static mix-training engineered by human. Evaluations on in-distribution (ID) and out-of-distribution (OOD) benchmarks reveal that RuleReasoner outperforms frontier LRMs by a significant margin (Delta4.1% on eight ID tasks and Delta10.4% on three OOD tasks over OpenAI-o1). Notably, our approach also exhibits higher computational efficiency compared to prior methods.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 74 of its 87 references that are in the map. First posted on arXiv 10 Jun 2025.

Builds on

74 of 87 references.

Closest ancestors · 14

Same direction, newest first

  1. 2025Curriculum Reinforcement Learning from Easy to Hard Tasks Improves LLM Reasoning7 Jun
  2. 2025On the Design of KL-Regularized Policy Gradient Algorithms for LLM Reasoning23 May
  3. 2025AceReason-Nemotron: Advancing Math and Code Reasoning through Reinforcement Learning22 May
  4. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Data6 May
  5. 2025TTRL: Test-Time Reinforcement Learning22 Apr
  6. 2025SRPO: A Cross-Domain Implementation of Large-Scale Reinforcement Learning on LLM19 Apr
  7. 2025Efficient Reinforcement Finetuning via Adaptive Curriculum Learning7 Apr
  8. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model31 Mar
  9. 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar
  10. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  11. 2025SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolution25 Feb
  12. 2025Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning20 Feb
2 more
  1. 2025LIMR: Less is More for RL Scaling17 Feb
  2. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 60

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  5. 2025Qwen3 Technical Reportcited by 8,195
  6. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  7. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  8. 2024Qwen2.5 Technical Reportcited by 4,713
  9. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  10. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  11. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  12. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
38 more
  1. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  2. 2024A Survey on LLM-as-a-Judgecited by 1,384
  3. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,057
  4. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 946
  5. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 655
  6. 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 641
  7. 2023DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretrainingcited by 385
  8. 2022Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thoughtcited by 334
  9. 2024Benchmarking Retrieval-Augmented Generation for Medicinecited by 326
  10. 2025Demystifying Long Chain-of-Thought Reasoning in LLMscited by 323
  11. 2020ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Languagecited by 309
  12. 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinkingcited by 308
  13. 2025MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learningcited by 294
  14. 2022FOLIO: Natural Language Reasoning with First-Order Logiccited by 233
  15. 2024A Survey on Data Selection for Language Modelscited by 230
  16. 2024RegMix: Data Mixture as Regression for Language Model Pre-trainingcited by 172
  17. 2019CLUTRR: A Diagnostic Benchmark for Inductive Reasoning from Textcited by 155
  18. 2025BIG-Bench Extra Hardcited by 123
  19. 2024Make Your LLM Fully Utilize the Contextcited by 118
  20. 2025Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoningcited by 118
  21. 2025Small Models Struggle to Learn from Strong Reasonerscited by 109
  22. 2024Faithful Logical Reasoning via Symbolic Chain-of-Thoughtcited by 99
  23. 2024Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervisioncited by 95
  24. 2023Benchmarking Large Language Models on CMExam -- A Comprehensive Chinese Medical Exam Datasetcited by 90
  25. 2023Testing the General Deductive Reasoning Capacity of Large Language Models Using OOD Examplescited by 84
  26. 2025Concise Reasoning via Reinforcement Learningcited by 78
  27. 2023Large Language Models can Learn Rulescited by 66
  28. 2023Large Language Models are In-Context Semantic Reasoners rather than Symbolic Reasonerscited by 63
  29. 2025Seek in the Dark: Reasoning via Test-Time Instance-Level Policy Gradient in Latent Spacecited by 40
  30. 2025How do language models learn facts? Dynamics, curricula and hallucinationscited by 33
  31. 2024Enhancing Reasoning Capabilities of LLMs via Principled Synthetic Logic Corpuscited by 29
  32. 2025Towards Reasoning Ability of Small Language Modelscited by 29
  33. 2024Can LLMs Reason with Rules? Logic Scaffolding for Stress-Testing and Improving LLMscited by 28
  34. 2025Large Language Models Meet Symbolic Provers for Logical Reasoning Evaluationcited by 27
  35. 2023ChatRule: Mining Logical Rules with Large Language Models for Knowledge Graph Reasoningcited by 26
  36. 2024RuleArena: A Benchmark for Rule-Guided Reasoning with LLMs in Real-World Scenarioscited by 26
  37. 2024IDEA: Enhancing the Rule Learning Ability of Large Language Model Agent through Induction, Deduction, and Abductioncited by 20
  38. 2024LogicGame: Benchmarking Rule-Based Reasoning Abilities of Large Language Modelscited by 19

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 3.

6 papers.

  1. 2025Understanding and Leveraging the Expert Specialization of Context Faithfulness in Mixture-of-Experts LLMscited by 12
  2. 2025Native Parallel Reasoner: Reasoning in Parallelism via Self-Distilled Reinforcement Learningcited by 12
  3. 2025LogicReward: Incentivizing LLM Reasoning via Step-Wise Logical Supervisioncited by 6
  4. 2026LM-Lexicon: Improving Definition Modeling via Harmonizing Semantic Expertscited by 0
  5. 2026SUPERNOVA: Eliciting General Reasoning in LLMs with Reinforcement Learning on Natural Instructionscited by 0
  6. 2026RISED: RubrIcs for agentic multi-environment Selection and sElf-Distillationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025ConciseRL: Conciseness-Guided Reinforcement Learning for Efficient Reasoning Models
  2. 2025Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective
  3. 2025JudgeLRM: Large Reasoning Models as a Judge
  4. 2025VerifyBench: A Systematic Benchmark for Evaluating Reasoning Verifiers Across Domains
  5. 2026Towards a Mechanistic Understanding of Large Reasoning Models: A Survey of Training, Inference, and Failures
  6. 2026Apriel-Reasoner: RL Post-Training for General-Purpose and Efficient Reasoning
  7. 2025Not All Tokens Matter: Towards Efficient LLM Reasoning via Token Significance in Reinforcement Learning
  8. 2025Motif-2-12.7B-Reasoning: A Practitioner's Guide to RL Training Recipes