mlsanityData to 7 Oct 2026

Prompt-R1: Collaborative Automatic Prompting Framework via End-to-end Reinforcement Learning

arXiv 2511.01016 ↗First posted 2 Nov 2025cs.CLcited by 7

Not on the map. Why?

Abstract (arXiv)

Recently, advanced large language models (LLMs) have emerged at an increasingly rapid pace. However, when faced with complex problems, most users are often unable to provide accurate and effective prompts to interact with LLMs, thus limiting the performance of LLMs. To address this challenge, we propose Prompt-R1, an end-to-end reinforcement learning framework that uses a small-scale LLM to collaborate with large-scale LLMs, replacing user interaction to solve problems better. This collaboration is cast as a multi-turn prompt interaction, where the small-scale LLM thinks and generates prompts, and the large-scale LLM performs complex reasoning. A dual-constrained reward is designed to optimize for correctness, generation quality, and reasoning accuracy. Prompt-R1 provides a plug-and-play framework that supports both inference and training with various large-scale LLMs. Experiments on multiple public datasets show that Prompt-R1 significantly outperforms baseline models across tasks. Our code is publicly available at https://github.com/QwenQKing/Prompt-R1.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 63 of its 71 references that are in the map. First posted on arXiv 2 Nov 2025.

Builds on

63 of 71 references.

References · 63

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  6. 2025Qwen3 Technical Reportcited by 8,195
  7. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  8. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  9. 2024GPT-4o System Cardcited by 4,467
  10. 2017Deep reinforcement learning from human preferencescited by 3,846
  11. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  12. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
38 more
  1. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  2. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  3. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  4. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  5. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  6. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
  7. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  8. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 893
  9. 2023Large Language Models as Optimizerscited by 830
  10. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 653
  11. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 628
  12. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 615
  13. 2024ORPO: Monolithic Preference Optimization without Reference Modelcited by 556
  14. 2023RRHF: Rank Responses to Align Language Models with Human Feedback without tearscited by 454
  15. 2025GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learningcited by 449
  16. 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMscited by 424
  17. 2023Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyondcited by 409
  18. 2025Agentic Context Engineering: Evolving Contexts for Self-Improving Language Modelscited by 349
  19. 2023Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discoverycited by 329
  20. 2020Transfer Learning in Deep Reinforcement Learning: A Surveycited by 259
  21. 2022Reasoning with Language Model Prompting: A Surveycited by 251
  22. 2024Reinforcement learningcited by 236
  23. 2021BookSum: A Collection of Datasets for Long-form Narrative Summarizationcited by 184
  24. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 153
  25. 2024A Comprehensive Survey of Small Language Models in the Era of Large Language Models: Techniques, Enhancements, Applications, Collaboration with LLMs, and Trustworthinesscited by 125
  26. 2024Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methodscited by 114
  27. 2023CoLLM: Integrating Collaborative Embeddings into Large Language Models for Recommendationcited by 99
  28. 2024RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMscited by 89
  29. 2025A Survey of LLM-based Agents in Medicine: How far are we from Baymax?cited by 84
  30. 2025PlanGenLLMs: A Modern Survey of LLM Planning Capabilitiescited by 82
  31. 2024Can GNN be Good Adapter for LLMs?cited by 71
  32. 2025HyperGraphRAG: Retrieval-Augmented Generation via Hypergraph-Structured Knowledge Representationcited by 68
  33. 2025Graph-R1: Towards Agentic GraphRAG Framework via End-to-end Reinforcement Learningcited by 50
  34. 2024RATT: A Thought Structure for Coherent and Correct LLM Reasoningcited by 39
  35. 2023Query-Dependent Prompt Evaluation and Optimization with Offline Inverse RLcited by 37
  36. 2023Prompt Optimization via Adversarial In-Context Learningcited by 33
  37. 2024CLAVE: An Adaptive Framework for Evaluating Values of LLM Generated Responsescited by 23
  38. 2025KBQA-o1: Agentic Knowledge Base Question Answering with Monte Carlo Tree Searchcited by 21

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 6.

7 papers.

  1. 2025LexGenius: An Expert-Level Benchmark for Large Language Models in Legal General Intelligencecited by 3
  2. 2026CoT2-Meta: Budgeted Metacognitive Control for Test-Time Reasoningcited by 2
  3. 2026MAXS: Meta-Adaptive Exploration with LLM Agentscited by 1
  4. 2026Language-based Trial and Error Falls Behind in the Era of Experiencecited by 1
  5. 2026p1: Better Prompt Optimization with Fewer Promptscited by 1
  6. 2026From Player to Master: Enhancing Test-Time Learning of LLM Agents via Reinforcement Learning over Memorycited by 1
  7. 2026Fully Unleashing the Multimodal Attacker: Meta-Adaptive Jailbreaking of Vision-Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026TAPR: Enhancing LLM Performance with a Task-Aware Prompt Rewriter
  2. 2024Reinforcement Learning for Aligning Large Language Models Agents with Interactive Environments: Quantifying and Mitigating Prompt Overfitting
  3. 2025Prompt reinforcing for long-term planning of large language models
  4. 2024QPO: Query-dependent Prompt Optimization via Multi-Loop Offline Reinforcement Learning
  5. 2025Can Prompt Difficulty be Online Predicted for Accelerating RL Finetuning of Reasoning Models?
  6. 2025PromptCoT 2.0: Scaling Prompt Synthesis for Large Language Model Reasoning
  7. 2026Prompting Policies for Multi-step Reasoning and Tool-Use in Black-box LLMs with Iterative Distillation of Experience
  8. 2024Towards Generalist Prompting for Large Language Models by Mental Models