SheetMind: Actions Set Accuracy, Agents Set the Failure Mode
On the map: LLM agents and tool use → Financial trading and forecasting agents
Abstract (arXiv)
Spreadsheet agents are converging on elaborate multi-agent designs, yet it is unclear how much of their performance comes from the agents rather than from the action interface they share. We answer this with SheetMind, a Manager-Action-Reflection framework, in a controlled study over all 221 tasks of the SheetCopilot Benchmark: five architectural variants, four backbones, exact McNemar tests on paired outcomes, and a checker reproducing the official chart and pivot comparisons. Replacing the high-level action API with primitive cell operations costs 47.1 points (p < 0.0001) and leaves the agent below a do-nothing baseline, whereas both extra agents together are worth 3.2 points: the Reflection Agent adds +4.5 (p = 0.013), the Manager +1.4 (p = 0.68). Decomposition instead changes how the system fails, cutting silently wrong outputs from 33% to 25% of tasks (p = 0.010). Capability saturates: GPT-5 and the five-times-cheaper GPT-5-mini are not significantly different (61.1% vs. 58.4%, p = 0.15), while GPT-3.5 loses 16.3 points and fails differently. A reflector must judge the step it just took, not the subtask. SheetMind reaches 61.1% Pass@1 with GPT-5 on the full SCB-221, against a do-nothing baseline of 9.0%. Accuracy comes from the operations an agent can name; the agents decide how it fails.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Financial trading and forecasting agents, one of 16 directions in LLM agents and tool use. Builds on 20 of its 25 references that are in the map. First posted on arXiv 14 Jun 2025.
Builds on
20 of 25 references.
References · 20
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
- 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,407
- 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
- 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
- 2025Why Do Multi-Agent LLM Systems Fail?cited by 604
- 2024Executable Code Actions Elicit Better LLM Agentscited by 564
- 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 558
- 2024Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perceptioncited by 293
8 more
- 2024OS-Copilot: Towards Generalist Computer Agents with Self-Improvementcited by 205
- 2024Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaborationcited by 189
- 2023ADaPT: As-Needed Decomposition and Planning with Language Modelscited by 177
- 2020With Little Power Comes Great Responsibilitycited by 142
- 2024AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agentscited by 114
- 2024SpreadsheetBench: Towards Challenging Real World Spreadsheet Manipulationcited by 111
- 2023SheetCopilot: Bringing Software Productivity to the Next Level through Large Language Modelscited by 57
- 2024MaxMind: A Memory Loop Network to Enhance Software Productivity based on Large Language Modelscited by 1
Built on by
Citing papers per year: 2025: 1; 2026: 2.
3 papers.
- 2025SheetBrain: A Neuro-Symbolic Agent for Accurate Reasoning over Complex and Large Spreadsheetscited by 8
- 2026Beyond Rows to Reasoning: Agentic Retrieval for Multimodal Spreadsheet Understanding and Editingcited by 4
- 2026Towards Robust Real-World Spreadsheet Understanding with Multi-Agent Multi-Format Reasoningcited by 2
- 2026Towards Robust Real-World Spreadsheet Understanding with Multi-Agent Multi-Format Reasoning14 Apr 2026
- 2026Beyond Rows to Reasoning: Agentic Retrieval for Multimodal Spreadsheet Understanding and Editing6 Mar 2026
- 2025SheetBrain: A Neuro-Symbolic Agent for Accurate Reasoning over Complex and Large Spreadsheets22 Oct 2025
Similar, not cited
Close in wording, no citation either way
- 2024SheetAgent: Towards A Generalist Agent for Spreadsheet Reasoning and Manipulation via Large Language Models
- 2026Evaluate the Stack, Not the Layer: Do Deterministic and LLM Gates for Agent Actions Fail Independently?
- 2026AgentAtlas: Beyond Outcome Leaderboards for LLM Agents
- 2026SheetCompass: Hierarchical Relation Graphs for Agentic Spreadsheet Reasoning
- 2025TableTalk: Scaffolding Spreadsheet Development with a Language Agent
- 2026SpreadsheetBench 2: Evaluating Agents on End-to-End Business Spreadsheet Workflows
- 2026Plans Work in Mysterious Ways: Evaluating a Plan Mode for Spreadsheet Agents
- 2026Beyond Resolution Rates: Behavioral Drivers of Coding Agent Success and Failure