A mixed policy to improve performance of language models on math problems
Not on the map. Why?
Abstract (arXiv)
When to solve math problems, most language models take a sampling strategy to predict next word according conditional probabilities. In the math reasoning step, it may generate wrong answer. Considering math problems are deterministic, we propose a mixed policy exploration approach to solve math problems with reinforcement learning. In peculiar, we propose a two level token exploration policy: the abstract level explores next token with probability and the second level is deterministic. Specifically, the abstract level policy will decide whether the token is operator or operand with probability sampling, while the second level is deterministic to select next token with the highest score in a greedy way. We test our method on GSM8K dataset with GPT-2 model, and demonstrate more than 2% performance gain. Our implementation is available at https://github.com/vividitytech/mathlmrl.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 9 of its 16 references that are in the map. First posted on arXiv 17 Jul 2023.
Builds on
9 of 16 references.
References · 9
- 2017Attention Is All You Needcited by 60,361
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2017Deep reinforcement learning from human preferencescited by 3,912
- 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
- 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,392
- 2022Solving math word problems with process- and outcome-based feedbackcited by 901
- 2023Solving Math Word Problems by Combining Language Models With Symbolic Solverscited by 124
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026DISPO: Enhancing Training Efficiency and Stability in Reinforcement Learning for Large Language Model Mathematical Reasoning
- 2025Diversity or Precision? A Deep Dive into Next Token Prediction
- 2025Ignore the KL Penalty! Boosting Exploration on Critical Tokens to Enhance RL Fine-Tuning
- 2025Big-Math: A Large-Scale, High-Quality Math Dataset for Reinforcement Learning in Language Models
- 2026Learning to Use Tools: Reinforcement Learning for Tool-Integrated Mathematical Reasoning
- 2026Improving Math Reasoning through Value-guided Informative Search
- 2026DIAL-OPD: Learning More from Fewer Tokens in On-Policy Distillation
- 2026On-Policy Delta Distillation for Multilingual Math Reasoning