mlsanityData to 7 Oct 2026

From AI Assistant to AI Scientist: Autonomous Discovery of LLM-RL Algorithms with LLM Agents

arXiv 2603.23951 ↗First posted 25 Mar 2026cs.CLcited by 2

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

Discovering improved policy optimization algorithms for language models remains a costly manual process requiring repeated mechanism-level modification and validation. Unlike simple combinatorial code search, this problem requires searching over algorithmic mechanisms tightly coupled with training dynamics while reusing empirical evidence across iterations. We propose POISE, a closed-loop framework for automated discovery of policy optimization algorithms for language models. POISE maintains a structured, genealogically linked archive linking proposals, executable implementations, standardized evaluations, and natural-language reflections to support evidence-driven iteration. In mathematical reasoning experiments starting from GRPO, POISE evaluates 64 candidate algorithms and discovers improved mechanisms, including analytic-variance scaling and validity masking. The best variant improves weighted Overall from 47.8 to 52.5 (+4.6) and increases AIME25 pass@32 from 26.7% to 43.3%, demonstrating the feasibility of automated policy optimization discovery while supporting interpretable design principles.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 18 of its 27 references that are in the map. First posted on arXiv 25 Mar 2026.

Builds on

18 of 27 references.

References · 18

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  5. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  6. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
  7. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,002
  8. 2024MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineeringcited by 429
  9. 2025Agent Laboratory: Using LLM Agents as Research Assistantscited by 405
  10. 2025The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Searchcited by 375
  11. 2024Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researcherscited by 348
  12. 2025AIDE: AI-Driven Exploration in the Space of Codecited by 206
6 more
  1. 2025Plan-and-Act: Improving Planning of Agents for Long-Horizon Taskscited by 190
  2. 2025Gold-medalist Performance in Solving Olympiad Geometry with AlphaGeometry2cited by 95
  3. 2025When Importance Sampling Misallocates Credit: Asymmetric Ratios for Outcome-Supervised RLcited by 21
  4. 2025Neural Architecture Discovery via Autonomous Evolutioncited by 15
  5. 2025SCI-IDEA: Context-Aware Scientific Ideation Using Token and Sentence Embeddingscited by 7
  6. 2025Hierarchically Encapsulated Representation for Protocol Design in Self-Driving Labscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loopscited by 25
  2. 2026Workflow Closure Is Not Scientific Closure in Auto-Research Systemscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026A First-Principles Derivation of LLM Policy Optimization: From Expected Reward to GRPO and Its Structural Extensions
  2. 2024Designing Algorithms Empowered by Language Models: An Analytical Framework, Case Studies, and Insights
  3. 2025On the Design of KL-Regularized Policy Gradient Algorithms for LLM Reasoning
  4. 2024Can Large Language Models Invent Algorithms to Improve Themselves?: Algorithm Discovery for Recursive Self-Improvement through Reinforcement Learning
  5. 2026NLPG: Natural-Language Policy Gradients for Self-Evolving Language Agents
  6. 2025Solver-Informed RL: Grounding Large Language Models for Authentic Optimization Modeling
  7. 2024Entropy-Regularized Token-Level Policy Optimization for Language Agent Reinforcement
  8. 2025Slow-Fast Policy Optimization: Reposition-Before-Update for LLM Reasoning