# Text Simplification by Tagging

arXiv: https://arxiv.org/abs/2103.05070 · HTML: https://mlsanity.com/paper/2103.05070/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 82 references that are in the map.
- First posted on arXiv 8 Mar 2021.

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Levenshtein Transformer](https://mlsanity.com/paper/1905.11006/) (2019)
- [GECToR -- Grammatical Error Correction: Tag, Not Rewrite](https://mlsanity.com/paper/2005.12592/) (2020)
- [Sentence Simplification with Deep Reinforcement Learning](https://mlsanity.com/paper/1703.10931/) (2017)
- [ASSET: A Dataset for Tuning and Evaluation of Sentence Simplification Models with Multiple Rewriting Transformations](https://mlsanity.com/paper/2005.00481/) (2020)
- [Encode, Tag, Realize: High-Precision Text Editing](https://mlsanity.com/paper/1909.01187/) (2019)
- [Parallel Iterative Edit Models for Local Sequence Transduction](https://mlsanity.com/paper/1910.02893/) (2019)
- [BLEU is Not Suitable for the Evaluation of Text Simplification](https://mlsanity.com/paper/1810.05995/) (2018)
- [Controllable Sentence Simplification](https://mlsanity.com/paper/1910.02677/) (2019)
- [EditNTS: An Neural Programmer-Interpreter Model for Sentence Simplification through Explicit Editing](https://mlsanity.com/paper/1906.08104/) (2019)
- [Felix: Flexible Text Editing Through Tagging and Insertion](https://mlsanity.com/paper/2003.10687/) (2020)
- [EASSE: Easier Automatic Sentence Simplification Evaluation](https://mlsanity.com/paper/1908.04567/) (2019)
- [Ensemble Distillation for Neural Machine Translation](https://mlsanity.com/paper/1702.01802/) (2017)
- [Unsupervised Neural Text Simplification](https://mlsanity.com/paper/1810.07931/) (2018)
- [Integrating Transformer and Paraphrase Rules for Sentence Simplification](https://mlsanity.com/paper/1810.11193/) (2018)
- [Dynamic Multi-Level Multi-Task Learning for Sentence Simplification](https://mlsanity.com/paper/1806.07304/) (2018)
- [Complexity-Weighted Loss and Diverse Reranking for Sentence Simplification](https://mlsanity.com/paper/1904.02767/) (2019)
- [MUSS: Multilingual Unsupervised Sentence Simplification by Mining Paraphrases](https://mlsanity.com/paper/2005.00352/) (2020)
- [Semi-Supervised Text Simplification with Back-Translation and Asymmetric Denoising Autoencoders](https://mlsanity.com/paper/2004.14693/) (2020)
- [Sentence Simplification with Memory-Augmented Neural Networks](https://mlsanity.com/paper/1804.07445/) (2018)
- [A Constrained Sequence-to-Sequence Neural Model for Sentence Simplification](https://mlsanity.com/paper/1704.02312/) (2017)
- [Improving Neural Text Simplification Model with Simplified Corpora](https://mlsanity.com/paper/1810.04428/) (2018)

## Built on by (most cited first, 27 of 27)

- [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://mlsanity.com/paper/2111.01243/) (2021)
- [Revisiting non-English Text Simplification: A Unified Multilingual Benchmark](https://mlsanity.com/paper/2305.15678/) (2023)
- [On the Ethical Considerations of Text Simplification](https://mlsanity.com/paper/2204.09565/) (2022)
- [Improving Iterative Text Revision by Learning Where to Edit from Other Revision Tasks](https://mlsanity.com/paper/2212.01350/) (2022)
- [Med-EASi: Finely Annotated Dataset and Models for Controllable Simplification of Medical Texts](https://mlsanity.com/paper/2302.09155/) (2023)
- [Deliberate then Generate: Enhanced Prompting Framework for Text Generation](https://mlsanity.com/paper/2305.19835/) (2023)
- [Teaching the Pre-trained Model to Generate Simple Texts for Text Simplification](https://mlsanity.com/paper/2305.12463/) (2023)
- [The SAMER Arabic Text Simplification Corpus](https://mlsanity.com/paper/2404.18615/) (2024)
- [Non-autoregressive Text Editing with Copy-aware Latent Alignments](https://mlsanity.com/paper/2310.07821/) (2023)
- [GRS: Combining Generation and Revision in Unsupervised Sentence Simplification](https://mlsanity.com/paper/2203.09742/) (2022)
- [Gender-Inclusive Grammatical Error Correction through Augmentation](https://mlsanity.com/paper/2306.07415/) (2023)
- [Opportunities and Challenges of LLMs in Education: An NLP Perspective](https://mlsanity.com/paper/2507.22753/) (2025)
- [Efficient Grammatical Error Correction Via Multi-Task Training and Optimized Training Schedule](https://mlsanity.com/paper/2311.11813/) (2023)
- [An LLM-Enhanced Adversarial Editing System for Lexical Simplification](https://mlsanity.com/paper/2402.14704/) (2024)
- [Revision for Concision: A Constrained Paraphrase Generation Task](https://mlsanity.com/paper/2210.14257/) (2022)
- [A New Dataset and Empirical Study for Sentence Simplification in Chinese](https://mlsanity.com/paper/2306.04188/) (2023)
- [Defining a New NLP Playground](https://mlsanity.com/paper/2310.20633/) (2023)
- [Context-Aware Document Simplification](https://mlsanity.com/paper/2305.06274/) (2023)
- [Introducing OmniGEC: A Silver Multilingual Dataset for Grammatical Error Correction](https://mlsanity.com/paper/2509.14504/) (2025)
- [Explain to me like I am five -- Sentence Simplification Using Transformers](https://mlsanity.com/paper/2212.04595/) (2022)
- [Automatic Correction of Human Translations](https://mlsanity.com/paper/2206.08593/) (2022)
- [Deepparse : An Extendable, and Fine-Tunable State-Of-The-Art Library for Parsing Multinational Street Addresses](https://mlsanity.com/paper/2311.11846/) (2023)
- [Edit-Constrained Decoding for Sentence Simplification](https://mlsanity.com/paper/2409.19247/) (2024)
- [SimplifyMyText: An LLM-Based System for Inclusive Plain Language Text Simplification](https://mlsanity.com/paper/2504.14223/) (2025)
- [Noisy Channel for Automatic Text Simplification](https://mlsanity.com/paper/2211.03152/) (2022)
- [SimCSum: Joint Learning of Simplification and Cross-lingual Summarization for Cross-lingual Science Journalism](https://mlsanity.com/paper/2304.01621/) (2023)
- [Can Knowledge Graphs Simplify Text?](https://mlsanity.com/paper/2308.06975/) (2023)

## Abstract (arXiv)

Edit-based approaches have recently shown promising results on multiple monolingual sequence transduction tasks. In contrast to conventional sequence-to-sequence (Seq2Seq) models, which learn to generate text from scratch as they are trained on parallel corpora, these methods have proven to be much more effective since they are able to learn to make fast and accurate transformations while leveraging powerful pre-trained language models. Inspired by these ideas, we present TST, a simple and efficient Text Simplification system based on sequence Tagging, leveraging pre-trained Transformer-based encoders. Our system makes simplistic data augmentations and tweaks in training and inference on a pre-existing system, which makes it less reliant on large amounts of parallel training data, provides more control over the outputs and enables faster inference speeds. Our best model achieves near state-of-the-art performance on benchmark test datasets for the task. Since it is fully non-autoregressive, it achieves faster inference speeds by over 11 times than the current state-of-the-art text simplification system.