mlsanityData to 7 Oct 2026

MulFeRL: Enhancing Reinforcement Learning with Verbal Feedback in a Multi-turn Loop

arXiv 2601.22900 ↗First posted 30 Jan 2026cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Reinforcement Learning with Verifiable Rewards (RLVR) is widely used to improve reasoning across domains, but outcome-only scalar rewards are often sparse and uninformative. This limitation is especially severe for failed samples, where scalar rewards indicate only that a solution is incorrect without explaining why the reasoning breaks down. In this paper, we leverage richer verbal feedback to guide RLVR on failed samples and convert feedback-induced progress into trainable learning signals. We propose MulFeRL (Multi-turn Feedback-guided Reinforcement Learning), a multi-turn, event-triggered RLVR framework that combines progress induction for feedback-guided regeneration of failed samples, progress credit assignment for learning from verifier-confirmed progress, and structured feedback injection for integrating feedback into the model's reasoning process. Trained on sampled OpenR1-Math, MulFeRL outperforms supervised, self-distillation-based, and RLVR baselines in-domain, while also showing strong out-of-domain generalization.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 36 of its 56 references that are in the map. First posted on arXiv 30 Jan 2026.

Builds on

36 of 56 references.

References · 36

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  6. 2023Let's Verify Step by Stepcited by 3,598
  7. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  8. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  9. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  10. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  11. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  12. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
24 more
  1. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,276
  2. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
  3. 2022Solving math word problems with process- and outcome-based feedbackcited by 871
  4. 2025Group Sequence Policy Optimizationcited by 788
  5. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 751
  6. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 672
  7. 2023Prometheus: Inducing Fine-grained Evaluation Capability in Language Modelscited by 558
  8. 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learningcited by 357
  9. 2026Reinforcement Learning via Self-Distillationcited by 285
  10. 2025Learning to Reason under Off-Policy Guidancecited by 266
  11. 2025Inference-Time Scaling for Generalist Reward Modelingcited by 233
  12. 2023TheoremQA: A Theorem-driven Question Answering datasetcited by 189
  13. 2025Crossing the Reward Bridge: Expanding RL with Verifiable Rewards Across Diverse Domainscited by 150
  14. 2023Chain of Hindsight Aligns Language Models with Feedbackcited by 147
  15. 2023Training Language Models with Language Feedback at Scalecited by 125
  16. 2019Learning from Dialogue after Deployment: Feed Yourself, Chatbot!cited by 102
  17. 2025Critique-GRPO: Advancing LLM Reasoning with Natural Language and Numerical Feedbackcited by 86
  18. 2025J1: Incentivizing Thinking in LLM-as-a-Judge via Reinforcement Learningcited by 73
  19. 2024Enhancing LLM Reasoning via Critique Models with Test-Time and Training-Time Supervisioncited by 60
  20. 2025Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifierscited by 40
  21. 2025A Survey of Process Reward Models: From Outcome Signals to Process Supervisions for Large Language Modelscited by 37
  22. 2025Beyond Correctness: Harmonizing Process and Outcome Rewards through RL Trainingcited by 29
  23. 2025Injecting External Knowledge into the Reasoning Process Enhances Retrieval-Augmented Generationcited by 3
  24. 2026Efficient Paths and Dense Rewards: Probabilistic Flow Reasoning for Large Language Modelscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Different Feedback, Different Updates: Selective Self-Learning from User Interactions for Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025ConfClip: Confidence-Weighted and Clipped Reward for Reinforcement Learning in LLMs
  2. 2025Extending RLVR to Open-Ended Tasks via Verifiable Multiple-Choice Reformulation
  3. 2025RLPR: Extrapolating RLVR to General Domains without Verifiers
  4. 2026Learning from Failures: Correction-Oriented Policy Optimization with Verifiable Rewards
  5. 2026Verifier-Induced Support Reshaping in On-Policy Optimization
  6. 2025Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs
  7. 2026SVR-R1: Bootstrapping Multi-modal Reasoning with Self-verification in Reinforcement Learning
  8. 2026Re2: Unlocking LLM Reasoning via Reinforcement Learning with Re-solving