# PRewrite: Prompt Rewriting with Reinforcement Learning

arXiv: https://arxiv.org/abs/2401.08189 · HTML: https://mlsanity.com/paper/2401.08189/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Placed in RL training of language and agent models, one of 15 directions in Deep reinforcement learning methods.
- Builds on 22 of its 25 references that are in the map.
- First posted on arXiv 16 Jan 2024.

## Placed in

- Deep reinforcement learning methods / [RL training of language and agent models](https://mlsanity.com/deeprl/rl-training-of-language-and-agent-models/)

## Builds on (most cited first, 22 of 22)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [Making Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2012.15723/) (2020)
- [Large Language Models Are Human-Level Prompt Engineers](https://mlsanity.com/paper/2211.01910/) (2022)
- [How Can We Know What Language Models Know?](https://mlsanity.com/paper/1911.12543/) (2019)
- [Universal Adversarial Triggers for Attacking and Analyzing NLP](https://mlsanity.com/paper/1908.07125/) (2019)
- [BARTScore: Evaluating Generated Text as Text Generation](https://mlsanity.com/paper/2106.11520/) (2021)
- [Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution](https://mlsanity.com/paper/2309.16797/) (2023)
- [RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning](https://mlsanity.com/paper/2205.12548/) (2022)
- [BERTese: Learning to Speak to BERT](https://mlsanity.com/paper/2103.05327/) (2021)
- [PADA: Example-based Prompt Learning for on-the-fly Adaptation to Unseen Domains](https://mlsanity.com/paper/2102.12206/) (2021)
- [Efficient (Soft) Q-Learning for Text Generation with Limited Good Data](https://mlsanity.com/paper/2106.07704/) (2021)

## Built on by (most cited first, 30 of 45)

- [A Systematic Survey of Automatic Prompt Optimization Techniques](https://mlsanity.com/paper/2502.16923/) (2025)
- [Efficient Prompting Methods for Large Language Models: A Survey](https://mlsanity.com/paper/2404.01077/) (2024)
- [Prompt Optimization with Human Feedback](https://mlsanity.com/paper/2405.17346/) (2024)
- [How to Train Your Advisor: Steering Black-Box LLMs with Advisor Models](https://mlsanity.com/paper/2510.02453/) (2025)
- [Zero-shot Video Moment Retrieval via Off-the-shelf Multimodal Large Language Models](https://mlsanity.com/paper/2501.07972/) (2025)
- [QPO: Query-dependent Prompt Optimization via Multi-Loop Offline Reinforcement Learning](https://mlsanity.com/paper/2408.10504/) (2024)
- [Probe-Rewrite-Evaluate: A Workflow for Reliable Benchmarks and Quantifying Evaluation Awareness](https://mlsanity.com/paper/2509.00591/) (2025)
- [Meta-Prompt Optimization for LLM-Based Sequential Decision Making](https://mlsanity.com/paper/2502.00728/) (2025)
- [Prompt4Trust: A Reinforcement Learning Prompt Augmentation Framework for Clinically-Aligned Confidence Calibration in Multimodal Large Language Models](https://mlsanity.com/paper/2507.09279/) (2025)
- [Enhancing the Capability and Robustness of Large Language Models through Reinforcement Learning-Driven Query Refinement](https://mlsanity.com/paper/2407.01461/) (2024)
- [Self-Instructed Derived Prompt Generation Meets In-Context Learning: Unlocking New Potential of Black-Box LLMs](https://mlsanity.com/paper/2409.01552/) (2024)
- [Irony Detection, Reasoning and Understanding in Zero-shot Learning](https://mlsanity.com/paper/2501.16884/) (2025)
- [TestAgent: Automatic Benchmarking and Exploratory Interaction for Evaluating LLMs in Vertical Domains](https://mlsanity.com/paper/2410.11507/) (2024)
- [PromptOptMe: Error-Aware Prompt Compression for LLM-based MT Evaluation Metrics](https://mlsanity.com/paper/2412.16120/) (2024)
- [A Survey on AgentOps: Categorization, Challenges, and Future Directions](https://mlsanity.com/paper/2508.02121/) (2025)
- [POET: Preference Optimization for Enhanced Text-to-Image Generation](https://mlsanity.com/paper/2510.12041/) (2025)
- [How to Auto-optimize Prompts for Domain Tasks? Adaptive Prompting and Reasoning through Evolutionary Domain Knowledge Adaptation](https://mlsanity.com/paper/2510.21148/) (2025)
- [Auto prompting without training labels: An LLM cascade for product quality assessment in e-commerce catalogs](https://mlsanity.com/paper/2510.23941/) (2025)
- [CodeCloak: A Method for Evaluating and Mitigating Code Leakage by LLM Code Assistants](https://mlsanity.com/paper/2404.09066/) (2024)
- [FedPOB: Sample-Efficient Federated Prompt Optimization via Bandits](https://mlsanity.com/paper/2509.24701/) (2025)
- [CAP: Controllable Alignment Prompting for Unlearning in LLMs](https://mlsanity.com/paper/2604.21251/) (2026)
- [S3-R1: Learning to Retrieve and Answer Step-by-Step with Synthetic Data](https://mlsanity.com/paper/2605.01248/) (2026)
- [Adapting Generalist Robot Policies with Semantic Reinforcement Learning](https://mlsanity.com/paper/2606.31958/) (2026)
- [Learning Personalized Prompts for Healthcare Guidance](https://mlsanity.com/paper/2412.15957/) (2024)
- [Meta-prompting Optimized Retrieval-augmented Generation](https://mlsanity.com/paper/2407.03955/) (2024)
- [Examples as the Prompt: A Scalable Approach for Efficient LLM Adaptation in E-Commerce](https://mlsanity.com/paper/2503.13518/) (2025)
- [Grammar-Guided Evolutionary Search for Discrete Prompt Optimisation](https://mlsanity.com/paper/2507.10326/) (2025)
- [Adaptive Multi-Agent Reasoning via Automated Workflow Generation](https://mlsanity.com/paper/2507.14393/) (2025)
- [Instance-level Randomization: Toward More Stable LLM Evaluations](https://mlsanity.com/paper/2509.12678/) (2025)
- [Learning to Rewrite Prompts for Bootstrapping LLMs on Downstream Tasks](https://mlsanity.com/paper/2510.06695/) (2025)

## Abstract (arXiv)

Prompt engineering is critical for the development of LLM-based applications. However, it is usually done manually in a "trial and error" fashion that can be time consuming, ineffective, and sub-optimal. Even for the prompts which seemingly work well, there is always a lingering question: can the prompts be made better with further modifications? To address these problems, we investigate automated prompt engineering in this paper. Specifically, we propose PRewrite, an automated method to rewrite an under-optimized prompt to a more effective prompt. We instantiate the prompt rewriter using a LLM. The rewriter LLM is trained using reinforcement learning to optimize the performance on a given downstream task. We conduct experiments on diverse benchmark datasets, which demonstrates the effectiveness of PRewrite.