mlsanityData to 7 Oct 2026

Nudging Beyond the Comfort Zone: Efficient Strategy-Guided Exploration for RLVR

arXiv 2605.15726 ↗First posted 15 May 2026cs.AI cs.CLcited by 4

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Reinforcement learning with verifiable rewards (RLVR) is a scalable paradigm for improving the mathematical reasoning of large language models, but it is fundamentally limited by exploration: the policy can only improve on trajectories it has already sampled. Sampling more rollouts alleviates this at prohibitive compute cost, while objective-level modifications offer little control over what is explored. We propose NudgeRL, a framework for structured, diversity-driven exploration in RLVR whose core component, Strategy Nudging, conditions each rollout on a lightweight strategy-level context, without requiring the context generator to solve the problem itself. To learn from such exploration, we decompose the advantage into inter- and intra-context terms and add a policy correction term that transfers discovered behaviors back to the base policy. Across five mathematical reasoning benchmarks, NudgeRL with 8 rollouts matches the strongest GRPO baseline using 32 rollouts with roughly 5 times fewer total tokens and 2.9 times less training compute. Our code is available at https://github.com/tally0818/NudgeRL.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 26 of its 35 references that are in the map. First posted on arXiv 15 May 2026.

Builds on

26 of 35 references.

Closest ancestors · 14

Same direction, newest first

  1. 2026Self-Hinting Language Models Enhance Reinforcement Learning3 Feb
  2. 2026POPE: Learning to Reason on Hard Problems via Privileged On-Policy Exploration26 Jan
  3. 2025Revisiting Entropy Regularization: Adaptive Coefficient Unlocks Its Potential for LLM Reinforcement Learning13 Oct
  4. 2025BroRL: Scaling Reinforcement Learning via Broadened Exploration1 Oct
  5. 2025Nudging the Boundaries of LLM Reasoning30 Sep
  6. 2025Decomposing the Entropy-Performance Exchange: The Missing Keys to Unlocking Effective Reinforcement Learning4 Aug
  7. 2025BREAD: Branched Rollouts from Expert Anchors Bridge SFT & RL for Reasoning20 Jun
  8. 2025ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models30 May
  9. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?18 Apr
  10. 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar
  11. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  12. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMs22 Jan
2 more
  1. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Training22 Nov
  2. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 12

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  4. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  5. 2024Qwen2.5-Coder Technical Reportcited by 1,151
  6. 2025Group Sequence Policy Optimizationcited by 788
  7. 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelscited by 755
  8. 2025MathArena: Evaluating LLMs on Uncontaminated Math Competitionscited by 275
  9. 2025Pass@K Policy Optimization: Solving Harder Reinforcement Learning Problemscited by 83
  10. 2025NVIDIA Nemotron 3: Efficient and Open Intelligencecited by 54
  11. 2026Expanding the Capabilities of Reinforcement Learning via Text Feedbackcited by 49
  12. 2026IsoCompute Playbook: Optimally Scaling Sampling Compute for LLM RLcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Instruction-Conditioned Exploration for Reinforcement Learning with Self-Distillation to an Unconditioned Policycited by 1
  2. 2026The Model Knows Another Way: Strategy Switching for Effective RLVR Explorationcited by 0
  3. 2026Mid-Harness: Scaling Actions Between Model and Harness for Terminal Agentscited by 0
  4. 2026Improving Math Reasoning through Value-guided Informative Searchcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026SAGE: Shaping Anchors for Guided Exploration in RLVR of LLMs
  2. 2026Decoupling Exploration from Optimization in RLVR
  3. 2025From Trial-and-Error to Improvement: A Systematic Analysis of LLM Exploration Mechanisms in RLVR
  4. 2025DeepSearch: Overcome the Bottleneck of Reinforcement Learning with Verifiable Rewards via Monte Carlo Tree Search
  5. 2026How You Begin is How You Reason: Driving Exploration in RLVR via Prefix-Tuned Priors
  6. 2026Controllable Exploration in Hybrid-Policy RLVR for Multi-Modal Reasoning
  7. 2025Depth-Breadth Synergy in RLVR: Unlocking LLM Reasoning Gains with Adaptive Exploration
  8. 2026DARL: Encouraging Diverse Answers for General Reasoning without Verifiers