mlsanityData to 7 Oct 2026

LLMs Corrupt Your Documents When You Delegate

arXiv 2604.15597 ↗First posted 17 Apr 2026cs.CL cs.HCcited by 9

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) are poised to disrupt knowledge work, with the emergence of delegated work as a new interaction paradigm (e.g., vibe coding). Delegation requires trust - the expectation that the LLM will faithfully execute the task without introducing errors into documents. We introduce DELEGATE-52 to study the readiness of AI systems in delegated workflows. DELEGATE-52 simulates long delegated workflows that require in-depth document editing across 52 professional domains, such as coding, crystallography, and music notation. Our large-scale experiment with 19 LLMs reveals that current models degrade documents during delegation: even frontier models (Gemini 3.1 Pro, Claude 4.6 Opus, GPT 5.4) corrupt an average of 25% of document content by the end of long workflows, with other models failing more severely. Additional experiments reveal that agentic tool use does not improve performance on DELEGATE-52, and that degradation severity is exacerbated by document size, length of interaction, or presence of distractor files. Our analysis shows that current LLMs are unreliable delegates: they introduce sparse but severe errors that silently corrupt documents, compounding over long interaction.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 70 of its 83 references that are in the map. First posted on arXiv 17 Apr 2026.

Builds on

70 of 83 references.

References · 70

  1. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  5. 2024GPT-4o System Cardcited by 4,467
  6. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  7. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  8. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  9. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  10. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
  11. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  12. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
38 more
  1. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
  2. 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 800
  3. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
  4. 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 656
  5. 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 358
  6. 2024WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?cited by 348
  7. 2024TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Taskscited by 306
  8. 2017Unsupervised Machine Translation Using Monolingual Corpora Onlycited by 272
  9. 2021Beyond Goldfish Memory: Long-Term Open-Domain Conversationcited by 266
  10. 2020Unsupervised Translation of Programming Languagescited by 262
  11. 2024Nomic Embed: Training a Reproducible Long Context Text Embeddercited by 255
  12. 2022Defining and Characterizing Reward Hackingcited by 184
  13. 2023GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Modelscited by 174
  14. 2025Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scalecited by 163
  15. 2023Self-Alignment with Instruction Backtranslationcited by 149
  16. 2025GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Taskscited by 142
  17. 2025xbench: Tracking Agents Productivity Scaling with Profession-Aligned Real-World Evaluationscited by 118
  18. 2025Which Economic Tasks are Performed with AI? Evidence from Millions of Claude Conversationscited by 88
  19. 2024Summary of a Haystack: A Challenge to Long-Context LLMs and RAG Systemscited by 87
  20. 2021Leveraging Automated Unit Tests for Unsupervised Code Translationcited by 82
  21. 2026MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Taskscited by 72
  22. 2024CRMArena: Understanding the Capacity of LLM Agents to Perform Professional CRM Tasks in Realistic Environmentscited by 69
  23. 2023CoEdIT: Text Editing by Task-Specific Instruction Tuningcited by 67
  24. 2025LifelongAgentBench: Evaluating LLM Agents as Lifelong Learnerscited by 65
  25. 2019Code Generation as a Dual Task of Code Summarizationcited by 62
  26. 2025Flipping the Dialogue: Training and Evaluating User Language Modelscited by 50
  27. 2024Can AI writing be salvaged? Mitigating Idiosyncrasies and Improving Human-AI Alignment in the Writing Process through Editscited by 49
  28. 2025ITBench: Evaluating AI Agents across Diverse Real-World IT Automation Taskscited by 48
  29. 2023Conversation Chronicles: Towards Diverse Temporal and Relational Dynamics in Multi-Session Conversationscited by 47
  30. 2023Can It Edit? Evaluating the Ability of Large Language Models to Follow Code Editing Instructionscited by 46
  31. 2025Future of Work with AI Agents: Auditing Automation and Augmentation Potential across the U.S. Workforcecited by 33
  32. 2023Beyond Accuracy: Evaluating Self-Consistency of Code Large Language Models with IdentityChaincited by 31
  33. 2022EditEval: An Instruction-Based Benchmark for Text Improvementscited by 30
  34. 2024SVGEditBench: A Benchmark Dataset for Quantitative Assessment of LLM's SVG Editing Capabilitiescited by 25
  35. 2025AI-Slop to AI-Polish? Aligning Language Models through Edit-Based Writing Rewards and Test-time Computationcited by 24
  36. 2024On Overcoming Miscalibrated Conversational Priors in LLM-based Chatbotscited by 23
  37. 2025StructEval: Benchmarking LLMs' Capabilities to Generate Structural Outputscited by 23
  38. 2024CodeEditorBench: Evaluating Code Editing Capability of Large Language Modelscited by 22

50 most cited of 70 All 70 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 9.

9 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026You Told Me to Do It: Measuring Instructional Text-induced Private Data Leakage in LLM Agents
  2. 2026Beyond Task Completion: Revealing Corrupt Success in LLM Agents through Procedure-Aware Evaluation
  3. 2024Coalitions of Large Language Models Increase the Robustness of AI Agents
  4. 2026Evaluation and Hardening of LLM System Instructions Against Extraction via Encoding Attacks
  5. 2025When Refusals Fail: Unstable Safety Mechanisms in Long-Context LLM Agents
  6. 2025ImpossibleBench: Measuring LLMs' Propensity of Exploiting Test Cases
  7. 2025Tool Preferences in Agentic LLMs are Unreliable
  8. 2025Artificial or Just Artful? Do LLMs Bend the Rules in Programming? ↗