mlsanityData to 7 Oct 2026

Reasoning-Enhanced Self-Training for Long-Form Personalized Text Generation

arXiv 2501.04167 ↗First posted 7 Jan 2025cs.CL cs.AI cs.IRcited by 26

On the map: LLM reasoning → Reinforcement learning for reasoning

also in 1 more
Abstract (arXiv)

Personalized text generation requires a unique ability of large language models (LLMs) to learn from context that they often do not encounter during their standard training. One way to encourage LLMs to better use personalized context for generating outputs that better align with the user's expectations is to instruct them to reason over the user's past preferences, background knowledge, or writing style. To achieve this, we propose Reasoning-Enhanced Self-Training for Personalized Text Generation (REST-PG), a framework that trains LLMs to reason over personal data during response generation. REST-PG first generates reasoning paths to train the LLM's reasoning abilities and then employs Expectation-Maximization Reinforced Self-Training to iteratively train the LLM based on its own high-reward outputs. We evaluate REST-PG on the LongLaMP benchmark, consisting of four diverse personalized long-form text generation tasks. Our experiments demonstrate that REST-PG achieves significant improvements over state-of-the-art baselines, with an average relative performance gain of 14.5% on the benchmark.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 16 of its 46 references that are in the map. First posted on arXiv 7 Jan 2025.

Builds on

16 of 46 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Search6 Jun

Other references · 15

  1. 2024Gemma: Open Models Based on Gemini Research and Technologycited by 889
  2. 2019Deep Learning Recommendation Model for Personalization and Recommendation Systemscited by 327
  3. 2023Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Mergingcited by 271
  4. 2024Hello Again! LLM-powered Personalized Agent for Long-term Dialoguecited by 121
  5. 2023Pearl: Personalizing Large Language Model Writing Assistants with Generation-Calibrated Retrieverscited by 91
  6. 2024LongLaMP: A Benchmark for Personalized Long-form Text Generationcited by 91
  7. 2022Self-Training: A Surveycited by 85
  8. 2024LLM-based NLG Evaluation: Current Status and Challengescited by 71
  9. 2024Personalized Pieces: Efficient Personalized Large Language Models through Collaborative Effortscited by 71
  10. 2023Teach LLMs to Personalize -- An Approach inspired by Writing Educationcited by 48
  11. 2024Leveraging Large Language Models for NLG Evaluation: Advances and Challengescited by 37
  12. 2024Comparing Retrieval-Augmentation and Parameter-Efficient Fine-Tuning for Privacy-Preserving Personalization of Large Language Modelscited by 20
3 more
  1. 2024Corporate Communication Companion (CCC): An LLM-empowered Writing Assistant for Workplace Social Mediacited by 14
  2. 2023A Survey on Large Language Models for Personalized and Explainable Recommendationscited by 11
  3. 2023Automated Evaluation of Personalized Text Generation using Large Language Modelscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 17; 2026: 9.

26 papers.

Sort:

By area:

  1. 2025A Survey of Personalized Large Language Models: Progress and Future Directionscited by 65
  2. 2025Measuring What Makes You Unique: Difference-Aware User Modeling for Enhancing LLM Personalizationcited by 44
  3. 2025LaMP-QA: A Benchmark for Personalized Long-form Question Answeringcited by 30
  4. 2025Think-While-Generating: On-the-Fly Reasoning for Personalized Long-Form Generationcited by 29
  5. 2025ExPerT: Effective and Explainable Evaluation of Personalized Long-Form Text Generationcited by 26
  6. 2025A Survey of Personalization: From RAG to Agentcited by 26
  7. 2025Latent Inter-User Difference Modeling for LLM Personalizationcited by 20
  8. 2025Instant Personalized Large Language Model Adaptation via Hypernetworkcited by 15
  9. 2025A Cognitive Writing Perspective for Constrained Long-Form Text Generationcited by 14
  10. 2025Personalized Safety in LLMs: A Benchmark and A Planning-Based Agent Approachcited by 14
  11. 2026A Survey on LLM-based Conversational User Simulationcited by 14
  12. 2025LongWriter-Zero: Mastering Ultra-Long Text Generation via Reinforcement Learningcited by 13
14 more
  1. 2026Toward Personalized LLM-Powered Agents: Foundations, Evaluation, and Future Directionscited by 10
  2. 2025Can LLMs Simulate Social Media Engagement? A Study on Action-Guided Response Generationcited by 8
  3. 2025SuperWriter: Reflection-Driven Long-Form Generation with Large Language Modelscited by 5
  4. 2025Learning from Natural Language Feedback for Personalized Question Answeringcited by 5
  5. 2025Pathways of Thoughts: Multi-Directional Thinking for Long-form Personalized Question Answeringcited by 5
  6. 2025Tagging the Thought: Unlocking Personalization Reasoning via Reinforcement Learningcited by 3
  7. 2026Rethinking Personalization in Large Language Models at the Token Levelcited by 3
  8. 2026Preference-Aware Rubric Learning for Personalized Evaluationcited by 2
  9. 2026Reasoning-Based Personalized Generation for Users with Sparse Datacited by 1
  10. 2025Plan-and-Refine: Diverse and Comprehensive Retrieval-Augmented Generationcited by 0
  11. 2026Improving User Privacy in Personalized Generation: Client-Side Retrieval-Augmented Modification of Server-Side Generated Speculationscited by 0
  12. 2026Sparse Personalized Text Generation with Multi-Trajectory Reasoningcited by 0
  13. 2026Psi-Bench: Evaluating Persona-Sensitive Influencing in Persuasive Dialoguescited by 0
  14. 2026LogicTree-RAG: Logic Tree-guided Retrieval-Augmented Generation for Long-form Patent Draftingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Reasoning Meets Personalization: Unleashing the Potential of Large Reasoning Model for Personalized Generation
  2. 2025A Model Can Help Itself: Reward-Free Self-Training for LLM Reasoning
  3. 2025PrLM: Learning Explicit Reasoning for Personalized RAG via Contrastive Reward Optimization
  4. 2024Large Language Models Can Self-Improve in Long-context Reasoning
  5. 2025Modular Techniques for Synthetic Long-Context Data Generation in Language Model Training and Evaluation
  6. 2023Large Language Models Cannot Self-Correct Reasoning Yet
  7. 2022Large Language Models Can Self-Improve
  8. 2025Self-Training Large Language Models with Confident Reasoning