# InsightBench: Evaluating Business Analytics Agents Through Multi-Step Insight Generation

arXiv: https://arxiv.org/abs/2407.06423 · HTML: https://mlsanity.com/paper/2407.06423/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use.
- Builds on 25 of its 56 references that are in the map.
- First posted on arXiv 8 Jul 2024.

## Placed in

- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 25 of 25)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback](https://mlsanity.com/paper/2305.14387/) (2023)
- [Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning](https://mlsanity.com/paper/1709.00103/) (2017)
- [FinQA: A Dataset of Numerical Reasoning over Financial Data](https://mlsanity.com/paper/2109.00122/) (2021)
- [Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models](https://mlsanity.com/paper/2305.04091/) (2023)
- [DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation](https://mlsanity.com/paper/2211.11501/) (2022)
- [InfiAgent-DABench: Evaluating Agents on Data Analysis Tasks](https://mlsanity.com/paper/2401.05507/) (2024)
- [OpenAgents: An Open Platform for Language Agents in the Wild](https://mlsanity.com/paper/2310.10634/) (2023)
- [CERT: Continual Pre-Training on Sketches for Library-Oriented Code Generation](https://mlsanity.com/paper/2206.06888/) (2022)
- [Natural Language to Code Generation in Interactive Data Science Notebooks](https://mlsanity.com/paper/2212.09248/) (2022)
- [Data-Copilot: Bridging Billions of Data and Humans with Autonomous Workflow](https://mlsanity.com/paper/2306.07209/) (2023)
- [Is GPT-4 a Good Data Analyst?](https://mlsanity.com/paper/2305.15038/) (2023)
- [MLCopilot: Unleashing the Power of Large Language Models in Solving Machine Learning Tasks](https://mlsanity.com/paper/2304.14979/) (2023)
- [From Words to Numbers: Your Large Language Model Is Secretly A Capable Regressor When Given In-Context Examples](https://mlsanity.com/paper/2404.07544/) (2024)
- [Training and Evaluating a Jupyter Notebook Data Science Assistant](https://mlsanity.com/paper/2201.12901/) (2022)
- [Benchmarking Data Science Agents](https://mlsanity.com/paper/2402.17168/) (2024)
- [Demonstration of InsightPilot: An LLM-Empowered Automated Data Exploration System](https://mlsanity.com/paper/2304.00477/) (2023)
- [Beyond Generating Code: Evaluating GPT on a Data Visualization Course](https://mlsanity.com/paper/2306.02914/) (2023)
- [Luna: An Evaluation Foundation Model to Catch Language Model Hallucinations with High Accuracy and Low Cost](https://mlsanity.com/paper/2406.00975/) (2024)
- [Capture the Flag: Uncovering Data Insights with Large Language Models](https://mlsanity.com/paper/2312.13876/) (2023)

## Built on by (most cited first, 30 of 32)

- [R&D-Agent: An LLM-Agent Framework Towards Autonomous Data Science](https://mlsanity.com/paper/2505.14738/) (2025)
- [Large Language Model-based Data Science Agent: A Survey](https://mlsanity.com/paper/2508.02744/) (2025)
- [HeurekaBench: A Benchmarking Framework for AI Co-scientist](https://mlsanity.com/paper/2601.01678/) (2026)
- [Data-to-Dashboard: Multi-Agent LLM Framework for Insightful Visualization in Enterprise Analytics](https://mlsanity.com/paper/2505.23695/) (2025)
- [Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap](https://mlsanity.com/paper/2608.05179/) (2026)
- [DataLab: A Unified Platform for LLM-Powered Business Intelligence](https://mlsanity.com/paper/2412.02205/) (2024)
- [DataCross: A Unified Benchmark and Agent Framework for Cross-Modal Heterogeneous Data Analysis](https://mlsanity.com/paper/2601.21403/) (2026)
- [DataSTORM: Deep Research on Large-Scale Databases using Exploratory Data Analysis and Data Storytelling](https://mlsanity.com/paper/2604.06474/) (2026)
- [AgentAda: Skill-Adaptive Data Analytics for Tailored Insight Discovery](https://mlsanity.com/paper/2504.07421/) (2025)
- [Hunt Instead of Wait: Evaluating Deep Data Research on Large Language Models](https://mlsanity.com/paper/2602.02039/) (2026)
- [Measuring Data Science Automation: A Survey of Evaluation Tools for AI Assistants and Agents](https://mlsanity.com/paper/2506.08800/) (2025)
- [InsightEval: An Expert-Curated Benchmark for Assessing Insight Discovery in LLM-Driven Data Agents](https://mlsanity.com/paper/2511.22884/) (2025)
- [MedInsightBench: Evaluating Medical Analytics Agents Through Multi-Step Insight Discovery in Multimodal Medical Data](https://mlsanity.com/paper/2512.13297/) (2025)
- [EvoOntology: A Self-Evolving Ontology Layer for Data Agents](https://mlsanity.com/paper/2609.15779/) (2026)
- [MDSF: Context-Aware Multi-Dimensional Data Storytelling Framework based on Large language Model](https://mlsanity.com/paper/2501.01014/) (2025)
- [Multi-Agent Multimodal Large Language Model Framework for Automated Interpretation of Fuel Efficiency Analytics in Public Transportation](https://mlsanity.com/paper/2511.13476/) (2025)
- [Beyond Description: A Multimodal Agent Framework for Insightful Chart Summarization](https://mlsanity.com/paper/2602.18731/) (2026)
- [Data Analysis in the Wild: Benchmarking Large Language Models Against Real-World Data Complexities](https://mlsanity.com/paper/2607.06482/) (2026)
- [Before the Action: Benchmarking LLMs on Prospective Hypothesis Discovery](https://mlsanity.com/paper/2607.15766/) (2026)
- [TraceBench: Controlled Evaluation of LLM Agents for Time-Series Root-Cause Attribution](https://mlsanity.com/paper/2608.27182/) (2026)
- [Tool-Aware Planning in Contact Center AI: Evaluating LLMs through Lineage-Guided Query Decomposition](https://mlsanity.com/paper/2602.14955/) (2026)
- [Behavioral Intelligence Platforms: From Event Streams to Autonomous Insight via Probabilistic Journey Graphs, Behavioral Knowledge Extraction, and Grounded Language Generation](https://mlsanity.com/paper/2604.22762/) (2026)
- [Towards Autonomous Business Intelligence via Data-to-Insight Discovery Agent](https://mlsanity.com/paper/2605.07202/) (2026)
- [Discovery Agents for Real-Time Analytics: Toward Proactive Insight Systems](https://mlsanity.com/paper/2605.27571/) (2026)
- [GRACE-DS: a Guarded Reward-guided Agent Correction Environment in Data Science](https://mlsanity.com/paper/2606.16000/) (2026)
- [AdaLens: Interactive Storyline for Monitoring and Steering Long-Running Agentic Data Analysis](https://mlsanity.com/paper/2608.17834/) (2026)
- [DI-Bench: Systematically Generating In-Domain Data Intelligence Benchmarks for Enterprise Agents](https://mlsanity.com/paper/2609.05776/) (2026)
- [InsightChain: Optimized Chain-of-Insight Analytics for LLM-driven Data Visualization](https://mlsanity.com/paper/2609.06438/) (2026)
- [DataCanvas-EDU: An Agentic Framework for Instructor-Guided Synthetic Data Generation in Business Analytics Education](https://mlsanity.com/paper/2609.19617/) (2026)
- [Argo-Bench: Evaluating Data Agents on Enterprise-Scale Workflows](https://mlsanity.com/paper/2610.02122/) (2026)

## Abstract (arXiv)

Data analytics is essential for extracting valuable insights from data that can assist organizations in making effective decisions. We introduce InsightBench, a benchmark dataset with three key features. First, it consists of 100 datasets representing diverse business use cases such as finance and incident management, each accompanied by a carefully curated set of insights planted in the datasets. Second, unlike existing benchmarks focusing on answering single queries, InsightBench evaluates agents based on their ability to perform end-to-end data analytics, including formulating questions, interpreting answers, and generating a summary of insights and actionable steps. Third, we conducted comprehensive quality assurance to ensure that each dataset in the benchmark had clear goals and included relevant and meaningful questions and analysis. Furthermore, we implement a two-way evaluation mechanism using LLaMA-3 as an effective, open-source evaluator to assess agents' ability to extract insights. We also propose AgentPoirot, our baseline data analysis agent capable of performing end-to-end data analytics. Our evaluation on InsightBench shows that AgentPoirot outperforms existing approaches (such as Pandas Agent) that focus on resolving single queries. We also compare the performance of open- and closed-source LLMs and various evaluation strategies. Overall, this benchmark serves as a testbed to motivate further development in comprehensive automated data analytics and can be accessed here: https://github.com/ServiceNow/insight-bench.