# DataSpace: Benchmarking Data Agents for Verifiable Analytics over Heterogeneous Workspaces

arXiv: https://arxiv.org/abs/2608.03451 · HTML: https://mlsanity.com/paper/2608.03451/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 56 references that are in the map.
- First posted on arXiv 4 Aug 2026.

## Builds on (most cited first, 30 of 31)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis](https://mlsanity.com/paper/2405.21075/) (2024)
- [Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task](https://mlsanity.com/paper/1809.08887/) (2018)
- [Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs](https://mlsanity.com/paper/2305.03111/) (2023)
- [Compositional Semantic Parsing on Semi-Structured Tables](https://mlsanity.com/paper/1508.00305/) (2015)
- [HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Data](https://mlsanity.com/paper/2004.07347/) (2020)
- [Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows](https://mlsanity.com/paper/2411.07763/) (2024)
- [MultiModalQA: Complex Question Answering over Text, Tables and Images](https://mlsanity.com/paper/2104.06039/) (2021)
- [The Dawn of Natural Language to SQL: Are We Fully Ready?](https://mlsanity.com/paper/2406.01265/) (2024)
- [DeepAnalyze: Agentic Large Language Models for Autonomous Data Science](https://mlsanity.com/paper/2510.16872/) (2025)
- [DABstep: Data Agent Benchmark for Multi-step Reasoning](https://mlsanity.com/paper/2506.23719/) (2025)
- [FinSQL: Model-Agnostic LLMs-based Text-to-SQL Framework for Financial Analysis](https://mlsanity.com/paper/2401.10506/) (2024)
- [NL2SQL-BUGs: A Benchmark for Detecting Semantic Errors in NL2SQL Translation](https://mlsanity.com/paper/2503.11984/) (2025)
- [DeepEye-SQL: A Software-Engineering-Inspired Text-to-SQL Framework](https://mlsanity.com/paper/2510.17586/) (2025)
- [AgenticData: An Agentic Data Analytics System for Heterogeneous Data](https://mlsanity.com/paper/2508.05002/) (2025)
- [IGenBench: Benchmarking the Reliability of Text-to-Infographic Generation](https://mlsanity.com/paper/2601.04498/) (2026)
- [FDABench: A Benchmark for Data Agents on Analytical Queries over Heterogeneous Data](https://mlsanity.com/paper/2509.02473/) (2025)
- [You Don't Know Until You Click:Automated GUI Testing for Production-Ready Software Evaluation](https://mlsanity.com/paper/2508.14104/) (2025)
- [Can AI Agents Answer Your Data Questions? A Benchmark for Data Agents](https://mlsanity.com/paper/2603.20576/) (2026)
- [DeepVIS: Bridging Natural Language and Data Visualization Through Step-wise Reasoning](https://mlsanity.com/paper/2508.01700/) (2025)
- [DeepEye: A Steerable Self-driving Data Agent System](https://mlsanity.com/paper/2603.28889/) (2026)
- [DataCross: A Unified Benchmark and Agent Framework for Cross-Modal Heterogeneous Data Analysis](https://mlsanity.com/paper/2601.21403/) (2026)
- [AgenticDataBench: A Comprehensive Benchmark for Data Agents](https://mlsanity.com/paper/2607.01647/) (2026)
- [ChartMark: A Structured Grammar for Chart Annotation](https://mlsanity.com/paper/2507.21810/) (2025)
- [LongDA: Benchmarking LLM Agents for Long-Document Data Analysis](https://mlsanity.com/paper/2601.02598/) (2026)
- [A Plug-and-Play Natural Language Rewriter for Natural Language to SQL](https://mlsanity.com/paper/2412.17068/) (2024)
- [DPC: Training-Free Text-to-SQL Candidate Selection via Dual-Paradigm Consistency](https://mlsanity.com/paper/2604.15163/) (2026)
- [Long-Document QA with Chain-of-Structured-Thought and Fine-Tuned SLMs](https://mlsanity.com/paper/2603.29232/) (2026)
- [TAIJI: MCP-based Multi-Modal Data Analytics on Data Lakes](https://mlsanity.com/paper/2505.11270/) (2025)

## Built on by (most cited first, 3 of 3)

- [DataMagic: Authoring Data Videos through Declarative Multi-Agent Orchestration](https://mlsanity.com/paper/2609.33403/) (2026)
- [Beyond Tables: Doc2DB-Bench for Relationally Faithful Document-to-Database Construction](https://mlsanity.com/paper/2608.08459/) (2026)
- [DataVista: Diagnosing Multimodal LLMs on Data Video Understanding](https://mlsanity.com/paper/2610.11993/) (2026)

## Abstract (arXiv)

Data agents enable natural-language analytics over organizational workspaces, where relevant evidence may be scattered across databases, structured files, long documents, and multimedia. Existing benchmarks largely isolate structured querying, retrieval, or open-ended analysis, leaving heterogeneous evidence discovery, complete tabular outputs, and deterministic evaluation insufficiently unified. We introduce DataSpace, a benchmark in which data agents produce verifiable tabular results from task-local heterogeneous workspaces. It contains 410 cross-language tasks and 7,439 artifacts totaling 15.01 GB across CSV, JSON, SQLite, Markdown, PDF, and video. DataSpace also served as the official evaluation benchmark for the KDD Cup 2026 Data Agents for Complex Data Analysis competition. Each agent receives only a question and workspace and returns the complete requested tabular result. We construct DataSpace with DataSpace-Builder, an execution-grounded framework comprising cross-language transformation, constraint-aware relational sampling, modality routing and artifact rendering, and human review and task repair by 11 domain experts. A deterministic evaluator performs header-invariant column alignment, type- and precision-aware normalization, and order-aware row comparison. Across six recently released frontier multimodal models and five widely used agent harnesses, the best accuracy reaches 66.34%, while harness choice creates a 15.36-point spread with the backbone fixed. Multimodal evidence integration and joins consistently reduce accuracy across all six backbones. These results show that DataSpace remains unsaturated and identify key challenges for improving data-agent reliability.