# Evaluating Factuality in Text Simplification

arXiv: https://arxiv.org/abs/2204.07562 · HTML: https://mlsanity.com/paper/2204.07562/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 52 references that are in the map.
- First posted on arXiv 15 Apr 2022.

## Builds on (most cited first, 28 of 28)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018)
- [On Faithfulness and Factuality in Abstractive Summarization](https://mlsanity.com/paper/2005.00661/) (2020)
- [Evaluating the Factual Consistency of Abstractive Text Summarization](https://mlsanity.com/paper/1910.12840/) (2019)
- [Asking and Answering Questions to Evaluate the Factual Consistency of Summaries](https://mlsanity.com/paper/2004.04228/) (2020)
- [FEQA: A Question Answering Evaluation Framework for Faithfulness Assessment in Abstractive Summarization](https://mlsanity.com/paper/2005.03754/) (2020)
- [Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality Metrics](https://mlsanity.com/paper/2104.13346/) (2021)
- [Neural Text Summarization: A Critical Evaluation](https://mlsanity.com/paper/1908.08960/) (2019)
- [Annotating and Modeling Fine-grained Factuality in Summarization](https://mlsanity.com/paper/2104.04302/) (2021)
- [Evaluating Factuality in Generation with Dependency-level Entailment](https://mlsanity.com/paper/2010.05478/) (2020)
- [Sentence Simplification with Deep Reinforcement Learning](https://mlsanity.com/paper/1703.10931/) (2017)
- [Decontextualization: Making Sentences Stand-Alone](https://mlsanity.com/paper/2102.05169/) (2021)
- [Controllable Sentence Simplification](https://mlsanity.com/paper/1910.02677/) (2019)
- [EditNTS: An Neural Programmer-Interpreter Model for Sentence Simplification through Explicit Editing](https://mlsanity.com/paper/1906.08104/) (2019)
- [Controllable Text Simplification with Explicit Paraphrasing](https://mlsanity.com/paper/2010.11004/) (2020)
- [Paragraph-level Simplification of Medical Texts](https://mlsanity.com/paper/2104.05767/) (2021)
- [Generating (Factual?) Narrative Summaries of RCTs: Experiments with Neural Multi-Document Summarization](https://mlsanity.com/paper/2008.11293/) (2020)
- [Keep it Simple: Unsupervised Simplification of Multi-Paragraph Text](https://mlsanity.com/paper/2107.03444/) (2021)
- [Dynamic Multi-Level Multi-Task Learning for Sentence Simplification](https://mlsanity.com/paper/1806.07304/) (2018)
- [Complexity-Weighted Loss and Diverse Reranking for Sentence Simplification](https://mlsanity.com/paper/1904.02767/) (2019)
- [Semi-Supervised Text Simplification with Back-Translation and Asymmetric Denoising Autoencoders](https://mlsanity.com/paper/2004.14693/) (2020)

## Built on by (most cited first, 30 of 34)

- [The Flan Collection: Designing Data and Methods for Effective Instruction Tuning](https://mlsanity.com/paper/2301.13688/) (2023)
- [LM vs LM: Detecting Factual Errors via Cross Examination](https://mlsanity.com/paper/2305.13281/) (2023)
- [Fine-grained Hallucination Detection and Editing for Language Models](https://mlsanity.com/paper/2401.06855/) (2024)
- [Editing Common Sense in Transformers](https://mlsanity.com/paper/2305.14956/) (2023)
- [I Don't Know: Explicit Modeling of Uncertainty with an [IDK] Token](https://mlsanity.com/paper/2412.06676/) (2024)
- [Retrieval augmentation of large language models for lay language generation](https://mlsanity.com/paper/2211.03818/) (2022)
- [Models See Hallucinations: Evaluating the Factuality in Video Captioning](https://mlsanity.com/paper/2303.02961/) (2023)
- [TrustScore: Reference-Free Evaluation of LLM Response Trustworthiness](https://mlsanity.com/paper/2402.12545/) (2024)
- [Do Text Simplification Systems Preserve Meaning? A Human Evaluation via Reading Comprehension](https://mlsanity.com/paper/2312.10126/) (2023)
- [Dancing Between Success and Failure: Edit-level Simplification Evaluation using SALSA](https://mlsanity.com/paper/2305.14458/) (2023)
- [NapSS: Paragraph-level Medical Text Simplification via Narrative Prompting and Sentence-matching Summarization](https://mlsanity.com/paper/2302.05574/) (2023)
- [Multilingual Simplification of Medical Texts](https://mlsanity.com/paper/2305.12532/) (2023)
- [SWiPE: A Dataset for Document-Level Simplification of Wikipedia Pages](https://mlsanity.com/paper/2305.19204/) (2023)
- [Teaching the Pre-trained Model to Generate Simple Texts for Text Simplification](https://mlsanity.com/paper/2305.12463/) (2023)
- [Converge to the Truth: Factual Error Correction via Iterative Constrained Editing](https://mlsanity.com/paper/2211.12130/) (2022)
- [Reference Matters: Benchmarking Factual Error Correction for Dialogue Summarization with Fine-grained Evaluation Framework](https://mlsanity.com/paper/2306.05119/) (2023)
- [A Survey of Automatic Hallucination Evaluation on Natural Language Generation](https://mlsanity.com/paper/2404.12041/) (2024)
- [WeCheck: Strong Factual Consistency Checker via Weakly Supervised Learning](https://mlsanity.com/paper/2212.10057/) (2022)
- [Clinical Insights: A Comprehensive Review of Language Models in Medicine](https://mlsanity.com/paper/2408.11735/) (2024)
- [Automatic and Human-AI Interactive Text Generation](https://mlsanity.com/paper/2310.03878/) (2023)
- [Evaluation Under Imperfect Benchmarks and Ratings: A Case Study in Text Simplification](https://mlsanity.com/paper/2504.09394/) (2025)
- [LLM-based Text Simplification and its Effect on User Comprehension and Cognitive Load](https://mlsanity.com/paper/2505.01980/) (2025)
- [Harnessing Knowledge and Reasoning for Human-Like Natural Language Generation: A Brief Review](https://mlsanity.com/paper/2212.03747/) (2022)
- [SimpLex: a lexical text simplification architecture](https://mlsanity.com/paper/2304.07002/) (2023)
- [Caught in the Web of Words: Do LLMs Fall for Spin in Medical Literature?](https://mlsanity.com/paper/2502.07963/) (2025)
- [Halluverse-M^3: A multitask multilingual benchmark for hallucination in LLMs](https://mlsanity.com/paper/2602.06920/) (2026)
- [Entity-Based Evaluation of Political Bias in Automatic Summarization](https://mlsanity.com/paper/2305.02321/) (2023)
- [Text and Audio Simplification: Human vs. ChatGPT](https://mlsanity.com/paper/2405.01592/) (2024)
- [Text Simplification of College Admissions Instructions: A Professionally Simplified and Verified Corpus](https://mlsanity.com/paper/2209.04529/) (2022)
- [A User-Centered Evaluation of Spanish Text Simplification](https://mlsanity.com/paper/2308.07556/) (2023)

## Abstract (arXiv)

Automated simplification models aim to make input texts more readable. Such methods have the potential to make complex information accessible to a wider audience, e.g., providing access to recent medical literature which might otherwise be impenetrable for a lay reader. However, such models risk introducing errors into automatically simplified texts, for instance by inserting statements unsupported by the corresponding original text, or by omitting key information. Providing more readable but inaccurate versions of texts may in many cases be worse than providing no such access at all. The problem of factual accuracy (and the lack thereof) has received heightened attention in the context of summarization models, but the factuality of automatically simplified texts has not been investigated. We introduce a taxonomy of errors that we use to analyze both references drawn from standard simplification datasets and state-of-the-art model outputs. We find that errors often appear in both that are not captured by existing evaluation metrics, motivating a need for research into ensuring the factual accuracy of automated simplification models.