LLMs Corrupt Your Documents When You Delegate
Not on the map. Why?
Abstract (arXiv)
Large Language Models (LLMs) are poised to disrupt knowledge work, with the emergence of delegated work as a new interaction paradigm (e.g., vibe coding). Delegation requires trust - the expectation that the LLM will faithfully execute the task without introducing errors into documents. We introduce DELEGATE-52 to study the readiness of AI systems in delegated workflows. DELEGATE-52 simulates long delegated workflows that require in-depth document editing across 52 professional domains, such as coding, crystallography, and music notation. Our large-scale experiment with 19 LLMs reveals that current models degrade documents during delegation: even frontier models (Gemini 3.1 Pro, Claude 4.6 Opus, GPT 5.4) corrupt an average of 25% of document content by the end of long workflows, with other models failing more severely. Additional experiments reveal that agentic tool use does not improve performance on DELEGATE-52, and that degradation severity is exacerbated by document size, length of interaction, or presence of distractor files. Our analysis shows that current LLMs are unreliable delegates: they introduce sparse but severe errors that silently corrupt documents, compounding over long interaction.
Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 70 of its 83 references that are in the map. First posted on arXiv 17 Apr 2026.
Builds on
70 of 83 references.
References · 70
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
- 2024GPT-4o System Cardcited by 4,467
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
- 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
- 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
38 more
- 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
- 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 800
- 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
- 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 656
- 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 358
- 2024WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?cited by 348
- 2024TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Taskscited by 306
- 2017Unsupervised Machine Translation Using Monolingual Corpora Onlycited by 272
- 2021Beyond Goldfish Memory: Long-Term Open-Domain Conversationcited by 266
- 2020Unsupervised Translation of Programming Languagescited by 262
- 2024Nomic Embed: Training a Reproducible Long Context Text Embeddercited by 255
- 2022Defining and Characterizing Reward Hackingcited by 184
- 2023GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Modelscited by 174
- 2025Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scalecited by 163
- 2023Self-Alignment with Instruction Backtranslationcited by 149
- 2025GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Taskscited by 142
- 2025xbench: Tracking Agents Productivity Scaling with Profession-Aligned Real-World Evaluationscited by 118
- 2025Which Economic Tasks are Performed with AI? Evidence from Millions of Claude Conversationscited by 88
- 2024Summary of a Haystack: A Challenge to Long-Context LLMs and RAG Systemscited by 87
- 2021Leveraging Automated Unit Tests for Unsupervised Code Translationcited by 82
- 2026MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Taskscited by 72
- 2024CRMArena: Understanding the Capacity of LLM Agents to Perform Professional CRM Tasks in Realistic Environmentscited by 69
- 2023CoEdIT: Text Editing by Task-Specific Instruction Tuningcited by 67
- 2025LifelongAgentBench: Evaluating LLM Agents as Lifelong Learnerscited by 65
- 2019Code Generation as a Dual Task of Code Summarizationcited by 62
- 2025Flipping the Dialogue: Training and Evaluating User Language Modelscited by 50
- 2024Can AI writing be salvaged? Mitigating Idiosyncrasies and Improving Human-AI Alignment in the Writing Process through Editscited by 49
- 2025ITBench: Evaluating AI Agents across Diverse Real-World IT Automation Taskscited by 48
- 2023Conversation Chronicles: Towards Diverse Temporal and Relational Dynamics in Multi-Session Conversationscited by 47
- 2023Can It Edit? Evaluating the Ability of Large Language Models to Follow Code Editing Instructionscited by 46
- 2025Future of Work with AI Agents: Auditing Automation and Augmentation Potential across the U.S. Workforcecited by 33
- 2023Beyond Accuracy: Evaluating Self-Consistency of Code Large Language Models with IdentityChaincited by 31
- 2022EditEval: An Instruction-Based Benchmark for Text Improvementscited by 30
- 2024SVGEditBench: A Benchmark Dataset for Quantitative Assessment of LLM's SVG Editing Capabilitiescited by 25
- 2025AI-Slop to AI-Polish? Aligning Language Models through Edit-Based Writing Rewards and Test-time Computationcited by 24
- 2024On Overcoming Miscalibrated Conversational Priors in LLM-based Chatbotscited by 23
- 2025StructEval: Benchmarking LLMs' Capabilities to Generate Structural Outputscited by 23
- 2024CodeEditorBench: Evaluating Code Editing Capability of Large Language Modelscited by 22
50 most cited of 70 All 70 (JSON)
Built on by
Citing papers per year: 2026: 9.
9 papers.
- 2026LLMs Get Lost in Evolving User Intentcited by 3
- 2026When Summaries Distort Decisions: Information Fidelity in LLM-Compressed Financial Analysiscited by 2
- 2026Synthetic Computers at Scale for Long-Horizon Productivity Simulationcited by 1
- 2026TSAssistant: A Human-in-the-Loop Agentic Framework for Automated Target Safety Assessmentcited by 0
- 2026GENIE: A Fine-Grained Measure for Noveltycited by 0
- 2026If It's Not Buggy, Don't Fix It: On the Dynamics of Iterative Bug-fixing with LLMscited by 0
- 2026"You're Right, Let Me Fix It": How LLM Agents Damage Correct Work When Falsely Accusedcited by 0
- 2026Trust the Brand, Lose Control: How Identity Hijacks LLM Agent Orchestrationcited by 0
- 2026Code Understanding is a Bottleneck for Coding Agentscited by 0
- 2026Code Understanding is a Bottleneck for Coding Agents7 Oct 2026
- 2026Trust the Brand, Lose Control: How Identity Hijacks LLM Agent Orchestration26 Sep 2026
- 2026"You're Right, Let Me Fix It": How LLM Agents Damage Correct Work When Falsely Accused26 Sep 2026
- 2026If It's Not Buggy, Don't Fix It: On the Dynamics of Iterative Bug-fixing with LLMs9 Sep 2026
- 2026LLMs Get Lost in Evolving User Intent22 Jul 2026
- 2026When Summaries Distort Decisions: Information Fidelity in LLM-Compressed Financial Analysis28 Jun 2026
- 2026GENIE: A Fine-Grained Measure for Novelty11 Jun 2026
- 2026Synthetic Computers at Scale for Long-Horizon Productivity Simulation30 Apr 2026
- 2026TSAssistant: A Human-in-the-Loop Agentic Framework for Automated Target Safety Assessment27 Apr 2026
Similar, not cited
Close in wording, no citation either way
- 2026You Told Me to Do It: Measuring Instructional Text-induced Private Data Leakage in LLM Agents
- 2026Beyond Task Completion: Revealing Corrupt Success in LLM Agents through Procedure-Aware Evaluation
- 2024Coalitions of Large Language Models Increase the Robustness of AI Agents
- 2026Evaluation and Hardening of LLM System Instructions Against Extraction via Encoding Attacks
- 2025When Refusals Fail: Unstable Safety Mechanisms in Long-Context LLM Agents
- 2025ImpossibleBench: Measuring LLMs' Propensity of Exploiting Test Cases
- 2025Tool Preferences in Agentic LLMs are Unreliable
- 2025Artificial or Just Artful? Do LLMs Bend the Rules in Programming? ↗