# Beyond the Chat: Executable and Verifiable Text-Editing with LLMs

arXiv: https://arxiv.org/abs/2309.15337 · HTML: https://mlsanity.com/paper/2309.15337/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 49 of its 115 references that are in the map.
- First posted on arXiv 27 Sep 2023.

## Builds on (most cited first, 30 of 49)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [On Faithfulness and Factuality in Abstractive Summarization](https://mlsanity.com/paper/2005.00661/) (2020)
- [A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity](https://mlsanity.com/paper/2302.04023/) (2023)
- [Enabling Large Language Models to Generate Text with Citations](https://mlsanity.com/paper/2305.14627/) (2023)
- [SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarization](https://mlsanity.com/paper/2111.09525/) (2021)
- [A Survey on Automated Fact-Checking](https://mlsanity.com/paper/2108.11896/) (2021)
- [Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translation](https://mlsanity.com/paper/2104.14478/) (2021)
- [Evaluating Verifiability in Generative Search Engines](https://mlsanity.com/paper/2304.09848/) (2023)
- [CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities](https://mlsanity.com/paper/2201.06796/) (2022)
- [QAFactEval: Improved QA-Based Factual Consistency Evaluation for Summarization](https://mlsanity.com/paper/2112.08542/) (2021)
- [Does Writing with Language Models Reduce Content Diversity?](https://mlsanity.com/paper/2309.05196/) (2023)
- [Co-Writing Screenplays and Theatre Scripts with Language Models: An Evaluation by Industry Professionals](https://mlsanity.com/paper/2209.14958/) (2022)
- [Neural Text Summarization: A Critical Evaluation](https://mlsanity.com/paper/1908.08960/) (2019)
- [Annotating and Modeling Fine-grained Factuality in Summarization](https://mlsanity.com/paper/2104.04302/) (2021)
- [A Recipe For Arbitrary Text Style Transfer with Large Language Models](https://mlsanity.com/paper/2109.03910/) (2021)
- [Automatically Neutralizing Subjective Bias in Text](https://mlsanity.com/paper/1911.09709/) (2019)
- [Decontextualization: Making Sentences Stand-Alone](https://mlsanity.com/paper/2102.05169/) (2021)
- [Evaluating Human-Language Model Interaction](https://mlsanity.com/paper/2212.09746/) (2022)
- [PEER: A Collaborative Language Model](https://mlsanity.com/paper/2208.11663/) (2022)
- [Sparks: Inspiration for Science Writing using Language Models](https://mlsanity.com/paper/2110.07640/) (2021)
- [Help me write a poem: Instruction Tuning as a Vehicle for Collaborative Poetry Writing](https://mlsanity.com/paper/2210.13669/) (2022)
- [CoEdIT: Text Editing by Task-Specific Instruction Tuning](https://mlsanity.com/paper/2305.09857/) (2023)
- [The Impact of Multiple Parallel Phrase Suggestions on Email Input and Composition Behaviour of Native and Non-Native English Writers](https://mlsanity.com/paper/2101.09157/) (2021)
- [Understanding Iterative Revision from Human-Written Text](https://mlsanity.com/paper/2203.03802/) (2022)
- [LENS: A Learnable Evaluation Metric for Text Simplification](https://mlsanity.com/paper/2212.09739/) (2022)
- [Read, Revise, Repeat: A System Demonstration for Human-in-the-loop Iterative Text Revision](https://mlsanity.com/paper/2204.03685/) (2022)
- [Supporting Human-AI Collaboration in Auditing LLMs with LLMs](https://mlsanity.com/paper/2304.09991/) (2023)

## Built on by (most cited first, 30 of 35)

- [Can AI writing be salvaged? Mitigating Idiosyncrasies and Improving Human-AI Alignment in the Writing Process through Edits](https://mlsanity.com/paper/2409.14509/) (2024)
- [AI-Slop to AI-Polish? Aligning Language Models through Edit-Based Writing Rewards and Test-time Computation](https://mlsanity.com/paper/2504.07532/) (2025)
- [The Impossibility of Fair LLMs](https://mlsanity.com/paper/2406.03198/) (2024)
- [Is Your Paper Being Reviewed by an LLM? Benchmarking AI Text Detection in Peer Review](https://mlsanity.com/paper/2502.19614/) (2025)
- [Co-Writing with AI, on Human Terms: Aligning Research with User Demands Across the Writing Process](https://mlsanity.com/paper/2504.12488/) (2025)
- [mEdIT: Multilingual Text Editing via Instruction Tuning](https://mlsanity.com/paper/2402.16472/) (2024)
- [Help Me Write a Story: Evaluating LLMs' Ability to Generate Writing Feedback](https://mlsanity.com/paper/2507.16007/) (2025)
- [Evaluating the Sensitivity of LLMs to Prior Context](https://mlsanity.com/paper/2506.00069/) (2025)
- [Steering AI-Driven Personalization of Scientific Text for General Audiences](https://mlsanity.com/paper/2411.09969/) (2024)
- [OnGoal: Tracking and Visualizing Conversational Goals in Multi-Turn Dialogue with Large Language Models](https://mlsanity.com/paper/2508.21061/) (2025)
- [Large Language Models Meet User Interfaces: The Case of Provisioning Feedback](https://mlsanity.com/paper/2404.11072/) (2024)
- [Spivavtor: An Instruction Tuned Ukrainian Text Editing Model](https://mlsanity.com/paper/2404.18880/) (2024)
- [HealthGenie: Empowering Users with Healthy Dietary Guidance through Knowledge Graph and Large Language Models](https://mlsanity.com/paper/2504.14594/) (2025)
- [HyperEdit: Unlocking Instruction-based Text Editing in LLMs via Hypernetworks](https://mlsanity.com/paper/2512.12544/) (2025)
- [Semantic Commit: Helping Users Update Intent Specifications for AI Memory at Scale](https://mlsanity.com/paper/2504.09283/) (2025)
- [Attribution Gradients: Incrementally Unfolding Citations for Critical Examination of Attributed AI Answers](https://mlsanity.com/paper/2510.00361/) (2025)
- [Large Language Models are Pattern Matchers: Editing Semi-Structured and Structured Documents with ChatGPT](https://mlsanity.com/paper/2409.07732/) (2024)
- [SummExecEdit: A Factual Consistency Benchmark in Summarization with Executable Edits](https://mlsanity.com/paper/2412.13378/) (2024)
- [Comparing Native and Non-native English Speakers' Behaviors in Collaborative Writing through Visual Analytics](https://mlsanity.com/paper/2502.18681/) (2025)
- [RevTogether: Supporting Science Story Revision with Multiple AI Agents](https://mlsanity.com/paper/2503.01608/) (2025)
- [CHOIR: A Chatbot-mediated Organizational Memory Leveraging Communication in University Research Labs](https://mlsanity.com/paper/2509.20512/) (2025)
- [CoLyricist: Enhancing Lyric Writing with AI through Workflow-Aligned Support](https://mlsanity.com/paper/2602.22606/) (2026)
- [Beyond the Desk: Barriers and Future Opportunities for AI to Assist Scientists in Embodied Physical Tasks](https://mlsanity.com/paper/2603.19504/) (2026)
- [Priming, Path-dependence, and Plasticity: Understanding the molding of user-LLM interaction and its implications from (many) chat logs in the wild](https://mlsanity.com/paper/2605.05767/) (2026)
- [How to Ask the AI: A User Perspective Survey for Large Language Model Prompting](https://mlsanity.com/paper/2608.07494/) (2026)
- [Voice Interaction With Conversational AI Could Facilitate Thoughtful Reflection and Substantive Revision in Writing](https://mlsanity.com/paper/2504.08687/) (2025)
- [Componentization: Decomposing Monolithic LLM Responses into Manipulable Semantic Units](https://mlsanity.com/paper/2509.08203/) (2025)
- [From Binary Groundedness to Support Relations: Towards a Reader-Centred Taxonomy for Comprehension of AI Output](https://mlsanity.com/paper/2604.08082/) (2026)
- [U-Define: Designing User Workflows for Hard and Soft Constraints in LLM-Based Planning](https://mlsanity.com/paper/2605.02765/) (2026)
- [StagedWorkspace: A Versioned Workspace for Knowledge-Work Agents](https://mlsanity.com/paper/2608.18050/) (2026)

## Abstract (arXiv)

Conversational interfaces powered by Large Language Models (LLMs) have recently become a popular way to obtain feedback during document editing. However, standard chat-based conversational interfaces do not support transparency and verifiability of the editing changes that they suggest. To give the author more agency when editing with an LLM, we present InkSync, an editing interface that suggests executable edits directly within the document being edited. Because LLMs are known to introduce factual errors, Inksync also supports a 3-stage approach to mitigate this risk: Warn authors when a suggested edit introduces new information, help authors Verify the new information's accuracy through external search, and allow an auditor to perform an a-posteriori verification by Auditing the document via a trace of all auto-generated content. Two usability studies confirm the effectiveness of InkSync's components when compared to standard LLM-based chat interfaces, leading to more accurate, more efficient editing, and improved user experience.