mlsanityData to 14 Oct 2026

SheetMind: Actions Set Accuracy, Agents Set the Failure Mode

arXiv 2506.12339 ↗First posted 14 Jun 2025cs.HC cs.AIcited by 3

On the map: LLM agents and tool use → Financial trading and forecasting agents

Abstract (arXiv)

Spreadsheet agents are converging on elaborate multi-agent designs, yet it is unclear how much of their performance comes from the agents rather than from the action interface they share. We answer this with SheetMind, a Manager-Action-Reflection framework, in a controlled study over all 221 tasks of the SheetCopilot Benchmark: five architectural variants, four backbones, exact McNemar tests on paired outcomes, and a checker reproducing the official chart and pivot comparisons. Replacing the high-level action API with primitive cell operations costs 47.1 points (p < 0.0001) and leaves the agent below a do-nothing baseline, whereas both extra agents together are worth 3.2 points: the Reflection Agent adds +4.5 (p = 0.013), the Manager +1.4 (p = 0.68). Decomposition instead changes how the system fails, cutting silently wrong outputs from 33% to 25% of tasks (p = 0.010). Capability saturates: GPT-5 and the five-times-cheaper GPT-5-mini are not significantly different (61.1% vs. 58.4%, p = 0.15), while GPT-3.5 loses 16.3 points and fails differently. A reflector must judge the step it just took, not the subtask. SheetMind reaches 61.1% Pass@1 with GPT-5 on the full SCB-221, against a do-nothing baseline of 9.0%. Accuracy comes from the operations an agent can name; the agents decide how it fails.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Financial trading and forecasting agents, one of 16 directions in LLM agents and tool use. Builds on 20 of its 25 references that are in the map. First posted on arXiv 14 Jun 2025.

Builds on

20 of 25 references.

References · 20

  1. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  2. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  3. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  4. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
  5. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,407
  6. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
  7. 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
  8. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
  9. 2025Why Do Multi-Agent LLM Systems Fail?cited by 604
  10. 2024Executable Code Actions Elicit Better LLM Agentscited by 564
  11. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 558
  12. 2024Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perceptioncited by 293
8 more
  1. 2024OS-Copilot: Towards Generalist Computer Agents with Self-Improvementcited by 205
  2. 2024Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaborationcited by 189
  3. 2023ADaPT: As-Needed Decomposition and Planning with Language Modelscited by 177
  4. 2020With Little Power Comes Great Responsibilitycited by 142
  5. 2024AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agentscited by 114
  6. 2024SpreadsheetBench: Towards Challenging Real World Spreadsheet Manipulationcited by 111
  7. 2023SheetCopilot: Bringing Software Productivity to the Next Level through Large Language Modelscited by 57
  8. 2024MaxMind: A Memory Loop Network to Enhance Software Productivity based on Large Language Modelscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2024SheetAgent: Towards A Generalist Agent for Spreadsheet Reasoning and Manipulation via Large Language Models
  2. 2026Evaluate the Stack, Not the Layer: Do Deterministic and LLM Gates for Agent Actions Fail Independently?
  3. 2026AgentAtlas: Beyond Outcome Leaderboards for LLM Agents
  4. 2026SheetCompass: Hierarchical Relation Graphs for Agentic Spreadsheet Reasoning
  5. 2025TableTalk: Scaffolding Spreadsheet Development with a Language Agent
  6. 2026SpreadsheetBench 2: Evaluating Agents on End-to-End Business Spreadsheet Workflows
  7. 2026Plans Work in Mysterious Ways: Evaluating a Plan Mode for Spreadsheet Agents
  8. 2026Beyond Resolution Rates: Behavioral Drivers of Coding Agent Success and Failure