mlsanityData to 14 Oct 2026

Dr Genre: Reinforcement Learning from Decoupled LLM Feedback for Generic Text Rewriting

arXiv 2503.06781 ↗First posted 9 Mar 2025cs.CL cs.AI cs.LGcited by 3

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

Generic text rewriting is a prevalent large language model (LLM) application that covers diverse real-world tasks, such as style transfer, fact correction, and email editing. These tasks vary in rewriting objectives (e.g., factual consistency vs. semantic preservation), making it challenging to develop a unified model that excels across all dimensions. Existing methods often specialize in either a single task or a specific objective, limiting their generalizability. In this work, we introduce a generic model proficient in factuality, stylistic, and conversational rewriting tasks. To simulate real-world user rewrite requests, we construct a conversational rewrite dataset, ChatRewrite, that presents ``natural''-sounding instructions, from raw emails using LLMs. Combined with other popular rewrite datasets, including LongFact for the factuality rewrite task and RewriteLM for the stylistic rewrite task, this forms a broad benchmark for training and evaluating generic rewrite models. To align with task-specific objectives, we propose Dr Genre, a Decoupled-reward learning framework for Generic rewriting, that utilizes objective-oriented reward models with a task-specific weighting. Evaluation shows that approach delivers higher-quality rewrites across all targeted tasks, improving objectives including instruction following (agreement), internal consistency (coherence), and minimal unnecessary edits (conciseness).

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 28 of its 40 references that are in the map. First posted on arXiv 9 Mar 2025.

Builds on

28 of 40 references.

Closest ancestors · 1

Same direction, newest first

  1. 2020Learning to summarize from human feedback2 Sep

Other references · 27

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2017Deep reinforcement learning from human preferencescited by 3,912
  6. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  7. 2015A large annotated corpus for learning natural language inferencecited by 1,847
  8. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 905
  9. 2022Large Language Models Can Self-Improvecited by 664
  10. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 625
  11. 2017Toward Controlled Generation of Textcited by 272
  12. 2017Style Transfer from Non-Parallel Text by Cross-Alignmentcited by 248
15 more
  1. 2023Fine-tuning Language Models for Factualitycited by 221
  2. 2018Dear Sir or Madam, May I introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfercited by 211
  3. 2018Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfercited by 191
  4. 2019Revisiting Self-Training for Neural Sequence Generationcited by 155
  5. 2024Foundational Autoraters: Taming Large Language Models for Better Automatic Evaluationcited by 74
  6. 2023RewriteLM: An Instruction-Tuned Large Language Model for Text Rewritingcited by 67
  7. 2024Formal-LLM: Integrating Formal Language and Natural Language for Controllable LLM-based Agentscited by 40
  8. 2020Parallel Data Augmentation for Formality Style Transfercited by 39
  9. 2021FRUIT: Faithfully Reflecting Updated Information in Textcited by 39
  10. 2022EdiT5: Semi-Autoregressive Text-Editing with T5 Warm-Startcited by 35
  11. 2020Unsupervised Paraphrasing via Deep Reinforcement Learningcited by 25
  12. 2024ChatGPT Based Data Augmentation for Improved Parameter-Efficient Debiasing of LLMscited by 18
  13. 2024Empowering Large Language Models for Textual Data Augmentationcited by 17
  14. 2023Don't Take This Out of Context! On the Need for Contextual Models and Evaluations for Stylistic Rewritingcited by 6
  15. 2021DRAG: Director-Generator Language Modelling Framework for Non-Parallel Author Stylized Rewritingcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025LeMix: Unified Scheduling for LLM Training and Inference on Multi-GPU Systemscited by 3
  2. 2025MACE: A Hybrid LLM Serving System with Colocated SLO-aware Continuous Retraining Alignmentcited by 2
  3. 2026PIMbot: A Self-Adaptive Attack Framework for Adversarial Manipulation of Multi-Robot Reinforcement Learningcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026Rewrite to Translate, Translate to Reward: Reinforcement Learning for Source Rewriting in Machine Translation
  2. 2023Towards an On-device Agent for Text Rewriting
  3. 2026DTO: a Differentiable Training Objective for Effective Counterfactual Story Rewriting
  4. 2026Teaching LLMs Human-Like Editing of Inappropriate Argumentation via Reinforcement Learning
  5. 2024LLM-based Rewriting of Inappropriate Argumentation using Reinforcement Learning from Machine Feedback
  6. 2025LongWriter-Zero: Mastering Ultra-Long Text Generation via Reinforcement Learning
  7. 2023LLMRefine: Pinpointing and Refining Large Language Models via Fine-Grained Actionable Feedback
  8. 2024Rewriting Conversational Utterances with Instructed Large Language Models