# Automatic and Human-AI Interactive Text Generation

arXiv: https://arxiv.org/abs/2310.03878 · HTML: https://mlsanity.com/paper/2310.03878/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 52 of its 91 references that are in the map.
- First posted on arXiv 5 Oct 2023.

## Builds on (most cited first, 30 of 52)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [Diffusion-LM Improves Controllable Text Generation](https://mlsanity.com/paper/2205.14217/) (2022)
- [Text Summarization with Pretrained Encoders](https://mlsanity.com/paper/1908.08345/) (2019)
- [DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models](https://mlsanity.com/paper/2210.08933/) (2022)
- [RARR: Researching and Revising What Language Models Say, Using Language Models](https://mlsanity.com/paper/2210.08726/) (2022)
- [Adversarial Example Generation with Syntactically Controlled Paraphrase Networks](https://mlsanity.com/paper/1804.06059/) (2018)
- [CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities](https://mlsanity.com/paper/2201.06796/) (2022)
- [Dear Sir or Madam, May I introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfer](https://mlsanity.com/paper/1803.06535/) (2018)
- [Reformulating Unsupervised Style Transfer as Paraphrase Generation](https://mlsanity.com/paper/2010.05700/) (2020)
- [Co-Writing with Opinionated Language Models Affects Users' Views](https://mlsanity.com/paper/2302.00560/) (2023)
- [Neural CRF Model for Sentence Alignment in Text Simplification](https://mlsanity.com/paper/2005.02324/) (2020)
- [Automatically Neutralizing Subjective Bias in Text](https://mlsanity.com/paper/1911.09709/) (2019)
- [Decontextualization: Making Sentences Stand-Alone](https://mlsanity.com/paper/2102.05169/) (2021)
- [ASSET: A Dataset for Tuning and Evaluation of Sentence Simplification Models with Multiple Rewriting Transformations](https://mlsanity.com/paper/2005.00481/) (2020)
- [PEER: A Collaborative Language Model](https://mlsanity.com/paper/2208.11663/) (2022)
- [Controllable Sentence Simplification](https://mlsanity.com/paper/1910.02677/) (2019)
- [RewriteLM: An Instruction-Tuned Large Language Model for Text Rewriting](https://mlsanity.com/paper/2305.15685/) (2023)
- [SeqDiffuSeq: Text Diffusion with Encoder-Decoder Transformers](https://mlsanity.com/paper/2212.10325/) (2022)
- [Felix: Flexible Text Editing Through Tagging and Insertion](https://mlsanity.com/paper/2003.10687/) (2020)
- [Understanding Iterative Revision from Human-Written Text](https://mlsanity.com/paper/2203.03802/) (2022)
- [Neural Syntactic Preordering for Controlled Paraphrase Generation](https://mlsanity.com/paper/2005.02013/) (2020)
- [Making Science Simple: Corpora for the Lay Summarisation of Scientific Literature](https://mlsanity.com/paper/2210.09932/) (2022)
- [LENS: A Learnable Evaluation Metric for Text Simplification](https://mlsanity.com/paper/2212.09739/) (2022)
- [Paragraph-level Simplification of Medical Texts](https://mlsanity.com/paper/2104.05767/) (2021)

## Built on by (most cited first, 4 of 4)

- [Pearl: Personalizing Large Language Model Writing Assistants with Generation-Calibrated Retrievers](https://mlsanity.com/paper/2311.09180/) (2023)
- [Can AI writing be salvaged? Mitigating Idiosyncrasies and Improving Human-AI Alignment in the Writing Process through Edits](https://mlsanity.com/paper/2409.14509/) (2024)
- [ReadCtrl: Personalizing text generation with readability-controlled instruction learning](https://mlsanity.com/paper/2406.09205/) (2024)
- [A Systematic Review of Data-to-Text NLG](https://mlsanity.com/paper/2402.08496/) (2024)

## Abstract (arXiv)

In this tutorial, we focus on text-to-text generation, a class of natural language generation (NLG) tasks, that takes a piece of text as input and then generates a revision that is improved according to some specific criteria (e.g., readability or linguistic styles), while largely retaining the original meaning and the length of the text. This includes many useful applications, such as text simplification, paraphrase generation, style transfer, etc. In contrast to text summarization and open-ended text completion (e.g., story), the text-to-text generation tasks we discuss in this tutorial are more constrained in terms of semantic consistency and targeted language styles. This level of control makes these tasks ideal testbeds for studying the ability of models to generate text that is both semantically adequate and stylistically appropriate. Moreover, these tasks are interesting from a technical standpoint, as they require complex combinations of lexical and syntactical transformations, stylistic control, and adherence to factual knowledge, -- all at once. With a special focus on text simplification and revision, this tutorial aims to provide an overview of the state-of-the-art natural language generation research from four major aspects -- Data, Models, Human-AI Collaboration, and Evaluation -- and to discuss and showcase a few significant and recent advances: (1) the use of non-retrogressive approaches; (2) the shift from fine-tuning to prompting with large language models; (3) the development of new learnable metric and fine-grained human evaluation framework; (4) a growing body of studies and datasets on non-English languages; (5) the rise of HCI+NLP+Accessibility interdisciplinary research to create real-world writing assistant systems.