# Learning to Rewrite Prompts for Personalized Text Generation

arXiv: https://arxiv.org/abs/2310.00152 · HTML: https://mlsanity.com/paper/2310.00152/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 34 references that are in the map.
- First posted on arXiv 29 Sep 2023.

## Builds on (most cited first, 19 of 19)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Making Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2012.15723/) (2020)
- [Large Language Models Are Human-Level Prompt Engineers](https://mlsanity.com/paper/2211.01910/) (2022)
- [How Can We Know What Language Models Know?](https://mlsanity.com/paper/1911.12543/) (2019)
- [Adafactor: Adaptive Learning Rates with Sublinear Memory Cost](https://mlsanity.com/paper/1804.04235/) (2018)
- [LaMP: When Large Language Models Meet Personalization](https://mlsanity.com/paper/2304.11406/) (2023)
- [RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning](https://mlsanity.com/paper/2205.12548/) (2022)
- [AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts](https://mlsanity.com/paper/2010.15980/) (2020)
- [Beam Search Strategies for Neural Machine Translation](https://mlsanity.com/paper/1702.01806/) (2017)
- [Training Millions of Personalized Dialogue Agents](https://mlsanity.com/paper/1809.01984/) (2018)
- [BERTese: Learning to Speak to BERT](https://mlsanity.com/paper/2103.05327/) (2021)
- [Less is More: Learning to Refine Dialogue History for Personalized Dialogue Generation](https://mlsanity.com/paper/2204.08128/) (2022)
- [Pchatbot: A Large-Scale Dataset for Personalized Chatbot](https://mlsanity.com/paper/2009.13284/) (2020)
- [Towards Controllable and Personalized Review Generation](https://mlsanity.com/paper/1910.03506/) (2019)
- [Knowledge-Enhanced Personalized Review Generation with Capsule Graph Neural Network](https://mlsanity.com/paper/2010.01480/) (2020)

## Built on by (most cited first, 30 of 40)

- [Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalization](https://mlsanity.com/paper/2406.01171/) (2024)
- [LongLaMP: A Benchmark for Personalized Long-form Text Generation](https://mlsanity.com/paper/2407.11016/) (2024)
- [A Survey of Personalized Large Language Models: Progress and Future Directions](https://mlsanity.com/paper/2502.11528/) (2025)
- [PersonaLens: A Benchmark for Personalization Evaluation in Conversational AI Assistants](https://mlsanity.com/paper/2506.09902/) (2025)
- [A Survey on Personalized Alignment -- The Missing Piece for Large Language Models in Real-World Applications](https://mlsanity.com/paper/2503.17003/) (2025)
- [LaMP-QA: A Benchmark for Personalized Long-form Question Answering](https://mlsanity.com/paper/2506.00137/) (2025)
- [User Embedding Model for Personalized Language Prompting](https://mlsanity.com/paper/2401.04858/) (2024)
- [A Survey on Personalized and Pluralistic Preference Alignment in Large Language Models](https://mlsanity.com/paper/2504.07070/) (2025)
- [A Survey of Personalization: From RAG to Agent](https://mlsanity.com/paper/2504.10147/) (2025)
- [ExPerT: Effective and Explainable Evaluation of Personalized Long-Form Text Generation](https://mlsanity.com/paper/2501.14956/) (2025)
- [Teaching Language Models to Evolve with Users: Dynamic Profile Modeling for Personalized Alignment](https://mlsanity.com/paper/2505.15456/) (2025)
- [Comparing Retrieval-Augmentation and Parameter-Efficient Fine-Tuning for Privacy-Preserving Personalization of Large Language Models](https://mlsanity.com/paper/2409.09510/) (2024)
- [On the Way to LLM Personalization: Learning to Remember User Conversations](https://mlsanity.com/paper/2411.13405/) (2024)
- [Learning Personalized Agents from Human Feedback](https://mlsanity.com/paper/2602.16173/) (2026)
- [WildIFEval: Instruction Following in the Wild](https://mlsanity.com/paper/2503.06573/) (2025)
- [Conversational User-AI Intervention: A Study on Prompt Rewriting for Improved LLM Response Generation](https://mlsanity.com/paper/2503.16789/) (2025)
- [T-POP: Test-Time Personalization with Online Preference Feedback](https://mlsanity.com/paper/2509.24696/) (2025)
- [ProCut: LLM Prompt Compression via Attribution Estimation](https://mlsanity.com/paper/2508.02053/) (2025)
- [Learning from Natural Language Feedback for Personalized Question Answering](https://mlsanity.com/paper/2508.10695/) (2025)
- [Role-Augmented Intent-Driven Generative Search Engine Optimization](https://mlsanity.com/paper/2508.11158/) (2025)
- [Long Context, Less Focus: A Scaling Gap in LLMs Revealed through Privacy and Personalization](https://mlsanity.com/paper/2602.15028/) (2026)
- [Navigating the Unknown: A Chat-Based Collaborative Interface for Personalized Exploratory Tasks](https://mlsanity.com/paper/2410.24032/) (2024)
- [Tuning-Free Personalized Alignment via Trial-Error-Explain In-Context Learning](https://mlsanity.com/paper/2502.08972/) (2025)
- [POET: Preference Optimization for Enhanced Text-to-Image Generation](https://mlsanity.com/paper/2510.12041/) (2025)
- [Evaluating Style-Personalized Text Generation: Challenges and Directions](https://mlsanity.com/paper/2508.06374/) (2025)
- [Learning Personalized Prompts for Healthcare Guidance](https://mlsanity.com/paper/2412.15957/) (2024)
- [Aligning VLM Assistants with Personalized Situated Cognition](https://mlsanity.com/paper/2506.00930/) (2025)
- [Preference-Aware Rubric Learning for Personalized Evaluation](https://mlsanity.com/paper/2605.31545/) (2026)
- [Iterative Critique-Refine Framework for Enhancing LLM Personalization](https://mlsanity.com/paper/2510.24469/) (2025)
- [Is Passive Expertise-Based Personalization Enough? A Case Study in AI-Assisted Test-Taking](https://mlsanity.com/paper/2511.23376/) (2025)

## Abstract (arXiv)

Facilitated by large language models (LLMs), personalized text generation has become a rapidly growing research direction. Most existing studies focus on designing specialized models for a particular domain, or they require fine-tuning the LLMs to generate personalized text. We consider a typical scenario in which the large language model, which generates personalized output, is frozen and can only be accessed through APIs. Under this constraint, all one can do is to improve the input text (i.e., text prompts) sent to the LLM, a procedure that is usually done manually. In this paper, we propose a novel method to automatically revise prompts for personalized text generation. The proposed method takes the initial prompts generated by a state-of-the-art, multistage framework for personalized generation and rewrites a few critical components that summarize and synthesize the personal context. The prompt rewriter employs a training paradigm that chains together supervised learning (SL) and reinforcement learning (RL), where SL reduces the search space of RL and RL facilitates end-to-end training of the rewriter. Using datasets from three representative domains, we demonstrate that the rewritten prompts outperform both the original prompts and the prompts optimized via supervised learning or reinforcement learning alone. In-depth analysis of the rewritten prompts shows that they are not only human readable, but also able to guide manual revision of prompts when there is limited resource to employ reinforcement learning to train the prompt rewriter, or when it is costly to deploy an automatic prompt rewriter for inference.