# InfoLossQA: Characterizing and Recovering Information Loss in Text Simplification

arXiv: https://arxiv.org/abs/2401.16475 · HTML: https://mlsanity.com/paper/2401.16475/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 79 references that are in the map.
- First posted on arXiv 29 Jan 2024.

## Builds on (most cited first, 30 of 42)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [ERASER: A Benchmark to Evaluate Rationalized NLP Models](https://mlsanity.com/paper/1911.03429/) (2019)
- [A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers](https://mlsanity.com/paper/2105.03011/) (2021)
- [SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarization](https://mlsanity.com/paper/2111.09525/) (2021)
- [Asking and Answering Questions to Evaluate the Factual Consistency of Summaries](https://mlsanity.com/paper/2004.04228/) (2020)
- [FEQA: A Question Answering Evaluation Framework for Faithfulness Assessment in Abstractive Summarization](https://mlsanity.com/paper/2005.03754/) (2020)
- [Learning to Ask: Neural Question Generation for Reading Comprehension](https://mlsanity.com/paper/1705.00106/) (2017)
- [Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluation](https://mlsanity.com/paper/2212.07981/) (2022)
- [Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of Perfect Information](https://mlsanity.com/paper/1805.04655/) (2018)
- [Decontextualization: Making Sentences Stand-Alone](https://mlsanity.com/paper/2102.05169/) (2021)
- [Towards Question-Answering as an Automatic Metric for Evaluating the Content Quality of a Summary](https://mlsanity.com/paper/2010.00490/) (2020)
- [Neural Question Generation from Text: A Preliminary Study](https://mlsanity.com/paper/1704.01792/) (2017)
- [DocNLI: A Large-scale Dataset for Document-level Natural Language Inference](https://mlsanity.com/paper/2106.09449/) (2021)
- [Generating Literal and Implied Subquestions to Fact-check Complex Claims](https://mlsanity.com/paper/2205.06938/) (2022)
- [Inferring Which Medical Treatments Work from Reports of Clinical Trials](https://mlsanity.com/paper/1904.01606/) (2019)
- [Paragraph-level Simplification of Medical Texts](https://mlsanity.com/paper/2104.05767/) (2021)
- [Stretching Sentence-pair NLI Models to Reason over Long Documents and Clusters](https://mlsanity.com/paper/2204.07447/) (2022)
- [Evidence Inference 2.0: More Data, Better Models](https://mlsanity.com/paper/2005.04177/) (2020)
- [Ask what's missing and what's useful: Improving Clarification Question Generation using Global Knowledge](https://mlsanity.com/paper/2104.06828/) (2021)
- [Summarizing, Simplifying, and Synthesizing Medical Evidence Using GPT-3 (with Varying Success)](https://mlsanity.com/paper/2305.06299/) (2023)
- [Controllable Open-ended Question Generation with A New Question Type Ontology](https://mlsanity.com/paper/2107.00152/) (2021)
- [Inquisitive Question Generation for High Level Text Comprehension](https://mlsanity.com/paper/2010.01657/) (2020)
- [On the Ethical Considerations of Text Simplification](https://mlsanity.com/paper/2204.09565/) (2022)
- [Discourse Level Factors for Sentence Deletion in Text Simplification](https://mlsanity.com/paper/1911.10384/) (2019)
- [Discourse Comprehension: A Question Answering Framework to Represent Sentence Connections](https://mlsanity.com/paper/2111.00701/) (2021)
- [SNaC: Coherence Error Detection for Narrative Summarization](https://mlsanity.com/paper/2205.09641/) (2022)
- [QUDEVAL: The Evaluation of Questions Under Discussion Discourse Parsing](https://mlsanity.com/paper/2310.14520/) (2023)

## Built on by (most cited first, 14 of 14)

- [Learning to Refine with Fine-Grained Natural Language Feedback](https://mlsanity.com/paper/2407.02397/) (2024)
- [FactPICO: Factuality Evaluation for Plain Language Summarization of Medical Evidence](https://mlsanity.com/paper/2402.11456/) (2024)
- [Behavioral Analysis of Information Salience in Large Language Models](https://mlsanity.com/paper/2502.14613/) (2025)
- [Do LLMs Understand Your Translations? Evaluating Paragraph-level MT with Question Answering](https://mlsanity.com/paper/2504.07583/) (2025)
- [Understanding Fine-grained Distortions in Reports of Scientific Findings](https://mlsanity.com/paper/2402.12431/) (2024)
- [Which questions should I answer? Salience Prediction of Inquisitive Questions](https://mlsanity.com/paper/2404.10917/) (2024)
- [AskQE: Question Answering as Automatic Evaluation for Machine Translation](https://mlsanity.com/paper/2504.11582/) (2025)
- [It Depends: Resolving Referential Ambiguity in Minimal Contexts with Commonsense Knowledge](https://mlsanity.com/paper/2509.16107/) (2025)
- [Simplifications are Absolutists: How Simplified Language Reduces Word Sense Awareness in LLM-Generated Definitions](https://mlsanity.com/paper/2507.11981/) (2025)
- [German4All -- A Dataset and Model for Readability-Controlled Paraphrasing in German](https://mlsanity.com/paper/2508.17973/) (2025)
- [Towards Trustworthy Lexical Simplification: Exploring Safety and Efficiency with Small LLMs](https://mlsanity.com/paper/2509.25086/) (2025)
- [Which course? Discourse! Teaching Discourse and Generation in the Era of LLMs](https://mlsanity.com/paper/2602.02878/) (2026)
- [Explain Like I'm 5 or Whatever I Choose: Evaluating the Potential for Interactive Complexity in Language Model Responses](https://mlsanity.com/paper/2606.06788/) (2026)
- [The Text Beside the Image: Detection, Utility and Leakage for Trustworthy Multimodal Medical Data and Beyond](https://mlsanity.com/paper/2609.33280/) (2026)

## Abstract (arXiv)

Text simplification aims to make technical texts more accessible to laypeople but often results in deletion of information and vagueness. This work proposes InfoLossQA, a framework to characterize and recover simplification-induced information loss in form of question-and-answer (QA) pairs. Building on the theory of Question Under Discussion, the QA pairs are designed to help readers deepen their knowledge of a text. We conduct a range of experiments with this framework. First, we collect a dataset of 1,000 linguist-curated QA pairs derived from 104 LLM simplifications of scientific abstracts of medical studies. Our analyses of this data reveal that information loss occurs frequently, and that the QA pairs give a high-level overview of what information was lost. Second, we devise two methods for this task: end-to-end prompting of open-source and commercial language models, and a natural language inference pipeline. With a novel evaluation framework considering the correctness of QA pairs and their linguistic suitability, our expert evaluation reveals that models struggle to reliably identify information loss and applying similar standards as humans at what constitutes information loss.