# EditPropBench: Measuring Factual Edit Propagation in Scientific Manuscripts

arXiv: https://arxiv.org/abs/2605.02083 · HTML: https://mlsanity.com/paper/2605.02083/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 27 references that are in the map.
- First posted on arXiv 3 May 2026.

## Builds on (most cited first, 25 of 25)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Mass-Editing Memory in a Transformer](https://mlsanity.com/paper/2210.07229/) (2022)
- [Evaluating the Factual Consistency of Abstractive Text Summarization](https://mlsanity.com/paper/1910.12840/) (2019)
- [Editing Factual Knowledge in Language Models](https://mlsanity.com/paper/2104.08164/) (2021)
- [Memory-Based Model Editing at Scale](https://mlsanity.com/paper/2206.06520/) (2022)
- [SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarization](https://mlsanity.com/paper/2111.09525/) (2021)
- [Editing Large Language Models: Problems, Methods, and Opportunities](https://mlsanity.com/paper/2305.13172/) (2023)
- [MT-Bench-101: A Fine-Grained Benchmark for Evaluating Large Language Models in Multi-Turn Dialogues](https://mlsanity.com/paper/2402.14762/) (2024)
- [Evaluating the Ripple Effects of Knowledge Editing in Language Models](https://mlsanity.com/paper/2307.12976/) (2023)
- [MultiChallenge: A Realistic Multi-Turn Conversation Evaluation Benchmark Challenging to Frontier LLMs](https://mlsanity.com/paper/2501.17399/) (2025)
- [A Comprehensive Study of Knowledge Editing for Large Language Models](https://mlsanity.com/paper/2401.01286/) (2024)
- [EasyEdit: An Easy-to-use Knowledge Editing Framework for Large Language Models](https://mlsanity.com/paper/2308.07269/) (2023)
- [MT-Eval: A Multi-Turn Capabilities Evaluation Benchmark for Large Language Models](https://mlsanity.com/paper/2401.16745/) (2024)
- [CoEdIT: Text Editing by Task-Specific Instruction Tuning](https://mlsanity.com/paper/2305.09857/) (2023)
- [WikiContradict: A Benchmark for Evaluating LLMs on Real-World Knowledge Conflicts from Wikipedia](https://mlsanity.com/paper/2406.13805/) (2024)
- [Long-form evaluation of model editing](https://mlsanity.com/paper/2402.09394/) (2024)
- [DUnE: Dataset for Unified Editing](https://mlsanity.com/paper/2311.16087/) (2023)
- [Diamonds in the Rough: Generating Fluent Sentences from Early-Stage Drafts for Academic Writing Assistance](https://mlsanity.com/paper/1910.09180/) (2019)
- [CASIMIR: A Corpus of Scientific Articles enhanced with Multiple Author-Integrated Revisions](https://mlsanity.com/paper/2403.00241/) (2024)
- [Langsmith: An Interactive Academic Text Revision System](https://mlsanity.com/paper/2010.04332/) (2020)
- [UniEdit: A Unified Knowledge Editing Benchmark for Large Language Models](https://mlsanity.com/paper/2505.12345/) (2025)
- [ParaRev: Building a dataset for Scientific Paragraph Revision annotated with revision instruction](https://mlsanity.com/paper/2501.05222/) (2025)
- [DocMEdit: Towards Document-Level Model Editing](https://mlsanity.com/paper/2505.19572/) (2025)
- [Identifying Reliable Evaluation Metrics for Scientific Text Revision](https://mlsanity.com/paper/2506.04772/) (2025)

## Built on by (most cited first, 1 of 1)

- [What Else Needs Fixing? Exploring Cost-Effective Test-Time Compute for Revision Propagation in Artifacts Generated Through Conversation](https://mlsanity.com/paper/2609.03254/) (2026)

## Abstract (arXiv)

Local factual edits in scientific manuscripts often create non-local revision obligations. If a dataset changes from 215 to 80 documents, claims such as 'medium-scale' or 'a few hundred items' may also become stale, even though they do not repeat the edited number. In an audit of recent arXiv cs.CL benchmark and dataset papers, we find fact-dependent qualitative claims in 37.2% of papers, suggesting that this dependency pattern is common in the target genre. We introduce EditPropBench, a benchmark for measuring whether LLM editors propagate factual edits through dependent manuscript claims. Each item contains an ML/NLP-style synthetic manuscript, a targeted edit, and a controlled fact graph with sentence-level labels for direct targets, required downstream updates, and unrelated text that should remain unchanged. We summarize cascade success with Edit-Ripple Adherence (ERA), the fraction of required downstream updates correctly revised, and validate the metric with adversarial probes and stress-test variants. On the hardest cases, where dependent claims use implicit or free-form wording rather than repeating the edited value, five LLM editing systems span ERA 0.148-0.705. Even the strongest misses roughly 30% of required cascade updates. This advantage persists in a mixed evaluation that includes easy cases solvable by deterministic substitution. EditPropBench shows that current LLM editors can repair many implicit consequences of factual edits, but reliable scientific revision still requires cascade-aware checking.