mlsanityData to 14 Oct 2026

From Reasoning to Code: GRPO Optimization for Underrepresented Languages

arXiv 2506.11027 ↗First posted 20 May 2025cs.LG cs.AI cs.PLcited by 11

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Generating accurate and executable code using Large Language Models (LLMs) remains a significant challenge for underrepresented programming languages, such as Prolog and Lisp, due to the scarcity of public training data compared to high-resource languages like Python. This paper introduces a generalizable Reinforcement Learning (RL) approach that combines small-scale versions of the Qwen2.5-Coder model with Group Relative Policy Optimization (GRPO) to enable effective code generation through reasoning. To address the limitations of sparse datasets, we integrate execution-driven feedback directly into the RL loop, utilizing a reward system that exploits both logical correctness and structural formatting. Experimental results on GSM8K dataset demonstrate significant improvements in reasoning quality and code accuracy across underrepresented languages. These findings underscore the potential of our approach to benefit a wide range of programming languages lacking extensive training resources by leveraging symbolic reasoning and interpreter-based feedback.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 17 of its 25 references that are in the map. First posted on arXiv 20 May 2025.

Builds on

17 of 25 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar

Other references · 16

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  6. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  7. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  8. 2024Qwen2.5-Coder Technical Reportcited by 1,203
  9. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,076
  10. 2023StarCoder: may the source be with you!cited by 799
  11. 2024A Survey on Large Language Models for Code Generationcited by 658
  12. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 358
4 more
  1. 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinkingcited by 313
  2. 2024RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learningcited by 151
  3. 2024Arithmetic Reasoning with LLM: Prolog Generation & Permutationcited by 17
  4. 2018Program Synthesis Through Reinforcement Learning Guided Tree Searchcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 7.

11 papers.

  1. 2025How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspectivecited by 43
  2. 2026CodeHacker: Automated Test Case Generation for Detecting Vulnerabilities in Competitive Programming Solutionscited by 8
  3. 2025Selective Expert Guidance for Effective and Diverse Exploration in Reinforcement Learning of LLMscited by 7
  4. 2026StaRPO: Stability-Augmented Reinforcement Policy Optimizationcited by 5
  5. 2026Find, Fix, Reason: Context Repair for Video Reasoningcited by 3
  6. 2026Structured Reasoning for Large Language Modelscited by 1
  7. 2026Agentic Monte Carlo: Simulating Reinforcement Learning for Black-Box Agentscited by 1
  8. 2026From Outcomes to Actions: Leveraging Hindsight for Long-Horizon Language Agent Trainingcited by 1
  9. 2025MoL-RL: Distilling Multi-Step Environmental Feedback into LLMs for Feedback-Independent Reasoningcited by 0
  10. 2025GAPO: Robust Advantage Estimation for Real-World Code LLMscited by 0
  11. 2026Improving Heart-Focused Medical Question Answering in LLMs via Variance-Aware Rubric Rewards with GRPOcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025CodeReasoner: Enhancing the Code Reasoning Ability with Reinforcement Learning
  2. 2025Afterburner: Reinforcement Learning Facilitates Self-Improving Code Efficiency Optimization
  3. 2026GRPO Training Dynamics for Small Language Models
  4. 2026BoT-GRPO: Efficient Process-Reward RL for Reasoning via Bag-of-Token Aggregation
  5. 2025MURPHY: Feedback-Aware GRPO with Retrospective Credit Assignment for Multi-Turn Code Generation
  6. 2025Scaf-GRPO: Scaffolded Group Relative Policy Optimization for Enhancing LLM Reasoning
  7. 2025G2RPO-A: Guided Group Relative Policy Optimization with Adaptive Guidance
  8. 2025Can GRPO Help LLMs Transcend Their Pretraining Origin?