mlsanityData to 7 Oct 2026

SpreadsheetBench: Towards Challenging Real World Spreadsheet Manipulation

arXiv 2406.14991 ↗First posted 21 Jun 2024cs.CL cs.SEcited by 100

Not on the map. Why?

Cited by 52 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce SpreadsheetBench, a challenging spreadsheet manipulation benchmark exclusively derived from real-world scenarios, designed to immerse current large language models (LLMs) in the actual workflow of spreadsheet users. Unlike existing benchmarks that rely on synthesized queries and simplified spreadsheet files, SpreadsheetBench is built from 912 real questions gathered from online Excel forums, which reflect the intricate needs of users. The associated spreadsheets from the forums contain a variety of tabular data such as multiple tables, non-standard relational tables, and abundant non-textual elements. Furthermore, we propose a more reliable evaluation metric akin to online judge platforms, where multiple spreadsheet files are created as test cases for each instruction, ensuring the evaluation of robust solutions capable of handling spreadsheets with varying values. Our comprehensive evaluation of various LLMs under both single-round and multi-round inference settings reveals a substantial gap between the state-of-the-art (SOTA) models and human performance, highlighting the benchmark's difficulty.

Key facts. Cited by 100 papers in the map (data through 7 Oct 2026). Builds on 30 of its 43 references that are in the map. First posted on arXiv 21 Jun 2024.

Builds on

30 of 43 references.

References · 30

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2023Qwen Technical Reportcited by 3,273
  4. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  5. 2024Mixtral of Expertscited by 1,785
  6. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  7. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  8. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 679
  9. 2015Compositional Semantic Parsing on Semi-Structured Tablescited by 509
  10. 2022DS-1000: A Natural and Reliable Benchmark for Data Science Code Generationcited by 442
  11. 2020TAPAS: Weakly Supervised Table Parsing via Pre-trainingcited by 432
  12. 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 402
18 more
  1. 2024OpenCodeInterpreter: Integrating Code Generation with Execution and Refinementcited by 242
  2. 2021TAPEX: Table Pre-training via Learning a Neural SQL Executorcited by 233
  3. 2022Binding Language Models in Symbolic Languagescited by 224
  4. 2021FeTaQA: Free-form Table Question Answeringcited by 174
  5. 2020Open Question Answering over Tables and Textcited by 163
  6. 2024InfiAgent-DABench: Evaluating Agents on Data Analysis Taskscited by 134
  7. 2021HiTab: A Hierarchical Table Dataset for Question Answering and Natural Language Generationcited by 117
  8. 2021Memorization vs. Generalization: Quantifying Data Leakage in NLP Performance Evaluationcited by 69
  9. 2023SheetCopilot: Bringing Software Productivity to the Next Level through Large Language Modelscited by 53
  10. 2021AIT-QA: Question Answering Dataset over Complex Tables in the Airline Industrycited by 44
  11. 2021SpreadsheetCoder: Formula Prediction from Semi-structured Contextcited by 41
  12. 2023Text2Analysis: A Benchmark of Table Question Answering with Advanced Data Analysis and Unclear Queriescited by 38
  13. 2024Tapilot-Crossing: Benchmarking and Evolving LLMs Towards Interactive Data Analysis Agentscited by 17
  14. 2023InstructExcel: A Benchmark for Natural Language Instruction in Excelcited by 14
  15. 2024FREB-TQA: A Fine-Grained Robustness Evaluation Benchmark for Table Question Answeringcited by 14
  16. 2024Auto-Formula: Recommend Formulas in Spreadsheets using Contrastive Learning for Table Representationscited by 12
  17. 2023LoFT: Enhancing Faithfulness and Diversity for Table-to-Text Generation via Logic Form Controlcited by 8
  18. 2017A Survey on Online Judge Systems and Their Applicationscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 17; 2026: 80.

100 papers.

Sort:

By area:

  1. 2026SkillOpt: Executive Strategy for Self-Evolving Agent Skillscited by 122
  2. 2025ComputerRL: Scaling End-to-End Online Reinforcement Learning for Computer Use Agentscited by 49
  3. 2026SkillGrad: Optimizing Agent Skills Like Gradient Descentcited by 23
  4. 2026From Raw Experience to Skill Consumption: A Systematic Study of Model-Generated Agent Skillscited by 18
  5. 2026SkillOpt-Lite: Better and Faster Agent Self-evolution via One Line of Vibecited by 18
  6. 2024PoTable: Towards Systematic Thinking via Plan-then-Execute Stage Reasoning on Tablescited by 13
  7. 2025MMTU: A Massive Multi-Task Table Understanding and Reasoning Benchmarkcited by 13
  8. 2024MiMoTable: A Multi-scale Spreadsheet Benchmark with Meta Operations for Table Reasoningcited by 12
  9. 2025RealHiTBench: A Comprehensive Realistic Hierarchical Table Benchmark for Evaluating LLM-Based Table Analysiscited by 12
  10. 2026SkillAxe: Sharpening LLM-Authored Agent Skills Through Evaluation-Guided Self-Refinementcited by 11
  11. 2026Skills on the Fly: Test-Time Adaptive Skill Synthesis for LLM Agentscited by 10
  12. 2025PPTArena: A Benchmark for PowerPoint Editingcited by 9
38 more
  1. 2025Finch: Benchmarking Finance & Accounting across Spreadsheet-Centric Enterprise Workflowscited by 9
  2. 2026ERNIE 5.0 Technical Reportcited by 9
  3. 2026SpreadsheetBench 2: Evaluating Agents on End-to-End Business Spreadsheet Workflowscited by 7
  4. 2026EnvHarness: Awakening Static Worlds for Agent Learningcited by 7
  5. 2026WikiSkill: Compiling Agent Experience into Persistent Knowledge for Skill Evolutioncited by 7
  6. 2025When Should Users Check? Modeling Confirmation Frequency in Multi-Step Agentic AI Taskscited by 6
  7. 2025SheetBrain: A Neuro-Symbolic Agent for Accurate Reasoning over Complex and Large Spreadsheetscited by 6
  8. 2026Agent Skill Evaluation and Evolution: Frameworks and Benchmarkscited by 6
  9. 2026HarnessEvolve: Learning from Reference Trajectories for Reliable Agent Self-Evolutioncited by 6
  10. 2025Text-to-Pipeline: Bridging Natural Language and Data Preparation Pipelinescited by 5
  11. 2025T2R-bench: A Benchmark for Generating Article-Level Reports from Real World Industrial Tablescited by 5
  12. 2026From Rows to Reasoning: A Retrieval-Augmented Multimodal Framework for Spreadsheet Understandingcited by 5
  13. 2026SkillCAT: Contrastive, Assessment-Augmented and Topology-AwareSkill Self-Evolution for LLM Agentscited by 5
  14. 2026AutoRefine: Compiling Trajectories into Validated Typed Agent Artifactscited by 4
  15. 2026ClawTrace: Cost-Aware Tracing for LLM Agent Skill Distillationcited by 4
  16. 2026MBABench: Evaluating LLM Agents on End-to-End Spreadsheet Tasks in Financecited by 4
  17. 2026Agent Skills Matter: Inferring Proprietary Skills from Execution Trajectoriescited by 4
  18. 2025Interactive Data Harmonization with LLM Agents: Opportunities and Challengescited by 3
  19. 2025Skill Discovery for Software Scripting Automation via Offline Simulations with LLMscited by 3
  20. 2025SheetMind: Actions Set Accuracy, Agents Set the Failure Modecited by 3
  21. 2025Empowering Tabular Data Preparation with Language Models: Why and How?cited by 3
  22. 2025Same Content, Different Representations: A Controlled Study for Table QAcited by 3
  23. 2025ConDABench: Interactive Evaluation of Language Models for Data Analysiscited by 3
  24. 2025Improving Language Agents through BREW: Bootstrapping expeRientially-learned Environmental knoWledgecited by 3
  25. 2026PrepBench: How Far Are We from Natural-Language-Driven Data Preparation?cited by 3
  26. 2026MedCUA-Bench: A Screenshot-Only Benchmark for Clinical Computer-Use Agentscited by 3
  27. 2026The Regression Tax: Decomposing Why Skills Help and Hurt LLM Agentscited by 3
  28. 2026Rethinking Self-Evolving Agent Skills: Feedback Dynamics over Multiple Roundscited by 3
  29. 2026Rethinking Self-Evolving Agents: Do We Still Need Prescribed Optimization Pipelines?cited by 3
  30. 2026SkillZip: Evaluation-Free Skill Compression for Self-Evolving Agents by Discovering Reusable Structurecited by 3
  31. 2026Harness-Zero: Harness Distillation via Agent-as-Harnesscited by 3
  32. 2026Agentic Tool Use in Large Language Modelscited by 2
  33. 2026ProCUA-SFT Technical Reportcited by 2
  34. 2026Workflow-Localized Mechanism Learning: Attribution-Guided Repair and Knowledge Reuse for Structured Agent Skillscited by 2
  35. 2026SKILL-KD: Contrastive Skill Distillation for LLM Agentscited by 2
  36. 2026SkillAdam: Stable and Efficient Skill Evolution for Agentscited by 2
  37. 2024STAND: Self-Aware Precondition Induction for Interactive Task Learningcited by 1
  38. 2026ReasonTabQA: A Comprehensive Benchmark for Table Question Answering from Real World Industrial Scenarioscited by 1

50 most cited of 100 All 100 (JSON)