# Large Language Models for Spreadsheets: Benchmarking Progress and Evaluating Performance with FLARE

arXiv: https://arxiv.org/abs/2506.17330 · HTML: https://mlsanity.com/paper/2506.17330/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Jun 2025.

## Built on by (most cited first, 1 of 1)

- [SpreadsheetBench 2: Evaluating Agents on End-to-End Business Spreadsheet Workflows](https://mlsanity.com/paper/2606.29955/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have demonstrated some significant capabilities across various domains; however, their effectiveness in spreadsheet related tasks remains underexplored. This study introduces a foundation for a comprehensive benchmark framework to evaluate the performance of leading LLMs in executing spreadsheet functions, formula generation and data manipulation tasks. The benchmark encompasses tasks ranging from basic formula creation to complex, real world spreadsheet scenarios. Our findings reveal that while LLMs exhibit proficiency in straightforward tasks, they often falter in complex, multi step operations, frequently producing plausible yet incorrect outputs. These results underscore the limitations of current LLMs in handling spreadsheet tasks that require precise logical reasoning and highlight the need for integrating symbolic reasoning capabilities into LLM architectures. To support this, we introduce FLARE (Formula Logic, Auditing, Reasoning and Evaluation) a new benchmark for evaluating LLM performance on real-world spreadsheet logic, auditing, and reasoning tasks.