mlsanityData to 7 Oct 2026

RewriteLM: An Instruction-Tuned Large Language Model for Text Rewriting

arXiv 2305.15685 ↗First posted 25 May 2023cs.CL cs.AIcited by 67

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have demonstrated impressive capabilities in creative tasks such as storytelling and E-mail generation. However, as LLMs are primarily trained on final text results rather than intermediate revisions, it might be challenging for them to perform text rewriting tasks. Most studies in the rewriting tasks focus on a particular transformation type within the boundaries of single sentences. In this work, we develop new strategies for instruction tuning and reinforcement learning to better align LLMs for cross-sentence rewriting tasks using diverse wording and structures expressed through natural languages including 1) generating rewriting instruction data from Wiki edits and public corpus through instruction generation and chain-of-thought prompting; 2) collecting comparison data for reward model training through a new ranking function. To facilitate this research, we introduce OpenRewriteEval, a novel benchmark covers a wide variety of rewriting types expressed through natural language instructions. Our results show significant improvements over a variety of baselines. The public repository is available on GitHub under Google Research (https://github.com/google-research/google-research/tree/master/rewritelm).

Key facts. Cited by 67 papers in the map (data through 7 Oct 2026). Builds on 43 of its 60 references that are in the map. First posted on arXiv 25 May 2023.

Builds on

43 of 60 references.

References · 43

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2017Inductive Representation Learning on Large Graphscited by 6,284
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  7. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  8. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  9. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  10. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  11. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  12. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
31 more
  1. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  2. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  3. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,304
  4. 2023PaLM 2 Technical Reportcited by 1,154
  5. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 896
  6. 2019Compressive Transformers for Long-Range Sequence Modellingcited by 699
  7. 2022Large Language Models Can Self-Improvecited by 656
  8. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 486
  9. 2018A Discourse-Aware Attention Model for Abstractive Summarization of Long Documentscited by 477
  10. 2019Multi-News: a Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Modelcited by 431
  11. 2021Efficient Attentions for Long Document Summarizationcited by 342
  12. 2018Dear Sir or Madam, May I introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfercited by 209
  13. 2019BIGPATENT: A Large-Scale Dataset for Abstractive and Coherent Summarizationcited by 155
  14. 2019Revisiting Self-Training for Neural Sequence Generationcited by 154
  15. 2022A Survey of Controllable Text Generation using Transformer-based Pre-trained Language Modelscited by 128
  16. 2019BillSum: A Corpus for Automatic Summarization of US Legislationcited by 117
  17. 2021A Recipe For Arbitrary Text Style Transfer with Large Language Modelscited by 107
  18. 2019Automatically Neutralizing Subjective Bias in Textcited by 105
  19. 2020ASSET: A Dataset for Tuning and Evaluation of Sentence Simplification Models with Multiple Rewriting Transformationscited by 101
  20. 2017JFLEG: A Fluency Corpus and Benchmark for Grammatical Error Correctioncited by 96
  21. 2022PEER: A Collaborative Language Modelcited by 86
  22. 2019This Email Could Save Your Life: Introducing the Task of Email Subject Line Generationcited by 58
  23. 2020Parallel Data Augmentation for Formality Style Transfercited by 39
  24. 2020TextSETTR: Few-Shot Text Style Extraction and Tunable Targeted Restylingcited by 39
  25. 2021FRUIT: Faithfully Reflecting Updated Information in Textcited by 39
  26. 2022EdiT5: Semi-Autoregressive Text-Editing with T5 Warm-Startcited by 35
  27. 2020Text Editing by Commandcited by 33
  28. 2022EditEval: An Instruction-Based Benchmark for Text Improvementscited by 30
  29. 2020Unsupervised Paraphrasing via Deep Reinforcement Learningcited by 25
  30. 2019Style Transfer for Texts: Retrain, Report Errors, Compare with Rewritescited by 19
  31. 2022Improving Wikipedia Verifiability with AIcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 7; 2024: 22; 2025: 29; 2026: 9.

67 papers.

Sort:

By area:

  1. 2023Statistical Rejection Sampling Improves Preference Optimizationcited by 321
  2. 2023MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedbackcited by 293
  3. 2023DRESS: Instructing Large Vision-Language Models to Align and Interact with Humans via Natural Language Feedbackcited by 99
  4. 2024LiPO: Listwise Preference Optimization through Learning-to-Rankcited by 94
  5. 2024A Design Space for Intelligent and Interactive Writing Assistantscited by 76
  6. 2024Aligning LLMs with Individual Preferences via Interactioncited by 63
  7. 2024Can AI writing be salvaged? Mitigating Idiosyncrasies and Improving Human-AI Alignment in the Writing Process through Editscited by 49
  8. 2024M4GT-Bench: Evaluation Benchmark for Black-Box Machine-Generated Text Detectioncited by 43
  9. 2024Large Language Model Inference Acceleration: A Comprehensive Hardware Perspectivecited by 36
  10. 2025Practical Poisoning Attacks against Retrieval-Augmented Generationcited by 35
  11. 2024SemEval-2024 Task 8: Multidomain, Multimodel and Multilingual Machine-Generated Text Detectioncited by 30
  12. 2024Benchmarking Large Language Models on Controllable Generation under Diversified Instructionscited by 26
38 more
  1. 2024Beyond Sparse Rewards: Enhancing Reinforcement Learning with Language Model Critique in Text Generationcited by 22
  2. 2024Prompt Public Large Language Models to Synthesize Data for Private On-device Applicationscited by 17
  3. 2024Waterfall: Framework for Robust and Scalable Text Watermarking and Provenance for LLMscited by 14
  4. 2025HACo-Det: A Study Towards Fine-Grained Machine-Generated Text Detection under Human-AI Coauthoringcited by 14
  5. 2024Montessori-Instruct: Generate Influential Training Data Tailored for Student Learningcited by 12
  6. 2024KIWI: A Dataset of Knowledge-Intensive Writing Instructions for Answering Research Questionscited by 11
  7. 2025TreeDiff: AST-Guided Code Generation with Diffusion LLMscited by 11
  8. 2025LLM-Friendly Knowledge Representation for Customer Supportcited by 11
  9. 2024A Piece of Theatre: Investigating How Teachers Design LLM Chatbots to Assist Adolescent Cyberbullying Educationcited by 10
  10. 2025Help Me Write a Story: Evaluating LLMs' Ability to Generate Writing Feedbackcited by 10
  11. 2023Towards an On-device Agent for Text Rewritingcited by 7
  12. 2023DEFT: Data Efficient Fine-Tuning for Pre-Trained Language Models via Unsupervised Core-Set Selectioncited by 6
  13. 2024Language Models as Critical Thinking Tools: A Case Study of Philosopherscited by 6
  14. 2023Back to the Future: Towards Explainable Temporal Reasoning with Large Language Modelscited by 5
  15. 2025Progressive Document-level Text Simplification via Large Language Modelscited by 5
  16. 2025What's Producible May Not Be Reachable: Measuring the Steerability of Generative Modelscited by 5
  17. 2025Role-Augmented Intent-Driven Generative Search Engine Optimizationcited by 5
  18. 2023Automatic and Human-AI Interactive Text Generationcited by 4
  19. 2024Proofread: Fixes All Errors with One Tapcited by 4
  20. 2024Audio Description Generation in the Era of LLMs and VLMs: A Review of Transferable Generative AI Technologiescited by 4
  21. 2025Neural Spelling: A Spell-Based BCI System for Language Neural Decodingcited by 4
  22. 2025Automatic Input Rewriting Improves Translation with Large Language Modelscited by 4
  23. 2025Curriculum-Guided Layer Scaling for Language Model Pretrainingcited by 4
  24. 2025Efficient Compositional Multi-tasking for On-device Large Language Modelscited by 4
  25. 2024Towards Reliable Detection of LLM-Generated Texts: A Comprehensive Evaluation Framework with CUDRTcited by 3
  26. 2024Large Language Model Should Understand Pinyin for Chinese ASR Error Correctioncited by 3
  27. 2025Efficient Layer-wise LLM Fine-tuning for Revision Intention Predictioncited by 3
  28. 2025K-Merge: Online Continual Merging of Adapters for On-device Large Language Modelscited by 3
  29. 2025MobileLLM-Pro Technical Reportcited by 3
  30. 2024AddrLLM: Address Rewriting via Large Language Model on Nationwide Logistics Datacited by 2
  31. 2025Dr Genre: Reinforcement Learning from Decoupled LLM Feedback for Generic Text Rewritingcited by 2
  32. 2025Ask2Loc: Learning to Locate Instructional Visual Answers by Asking Questionscited by 2
  33. 2025HyperEdit: Unlocking Instruction-based Text Editing in LLMs via Hypernetworkscited by 2
  34. 2024Hard-Synth: Synthesizing Diverse Hard Samples for ASR using Zero-Shot TTS and LLMcited by 1
  35. 2025SEval-Ex: A Statement-Level Framework for Explainable Summarization Evaluationcited by 1
  36. 2025CIE: Controlling Language Model Text Generations Using Continuous Signalscited by 1
  37. 2025LLM Unlearning on Noisy Forget Sets: A Study of Incomplete, Rewritten, and Watermarked Datacited by 1
  38. 2024Weak Reward Model Transforms Generative Models into Robust Causal Event Extraction Systemscited by 0

50 most cited of 67 All 67 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Towards Alignment-Centric Paradigm: A Survey of Instruction Tuning in Large Language Models
  2. 2025LongWriter-Zero: Mastering Ultra-Long Text Generation via Reinforcement Learning
  3. 2024Rewriting Conversational Utterances with Instructed Large Language Models
  4. 2026From Coarse to Fine: Benchmarking and Reward Modeling for Writing-Centric Generation Tasks
  5. 2025REFINE-AF: A Task-Agnostic Framework to Align Language Models via Self-Generated Instructions using Reinforcement Learning from Automated Feedback
  6. 2023LLMRefine: Pinpointing and Refining Large Language Models via Fine-Grained Actionable Feedback
  7. 2023CITING: Large Language Models Create Curriculum for Instruction Tuning
  8. 2026CreativeInstruct: Scalably Teaching LLMs to Balance Quality, Creativity, and Diversity