SpatialBench: Can Agents Analyze Real-World Spatial Biology Data?
Not on the map. Why?
Abstract (arXiv)
Spatial transcriptomics assays are rapidly increasing in scale and complexity, making computational analysis a major bottleneck in biological discovery. Although frontier AI agents have improved dramatically at software engineering and general data analysis, it remains unclear whether they can extract biological insight from messy, real-world spatial datasets. We introduce SpatialBench, a benchmark of 146 verifiable problems derived from practical spatial analysis workflows spanning five spatial technologies and seven task categories. Each problem provides a snapshot of experimental data immediately prior to an analysis step and a deterministic grader that evaluates recovery of a key biological result. Benchmark data on frontier models shows that base model accuracy remains low (20-38% across model families), with strong model-task and model-platform interactions. Harness design has a large empirical effect on performance, indicating that tools, prompts, control flow, and execution environment should be evaluated and improved as first-class objects. SpatialBench serves both as a measurement tool and a diagnostic lens for developing agents that can interact with real spatial datasets faithfully, transparently, and reproducibly.
Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 7 of its 17 references that are in the map. First posted on arXiv 26 Dec 2025.
Builds on
7 of 17 references.
References · 7
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
- 2022Large Language Models Encode Clinical Knowledgecited by 1,830
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
- 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
- 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,288
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026SP-Mind: An Autonomous Reasoning Agent for Spatial Proteomics Analysiscited by 4
- 2026Verifiable Benchmarking of Long-Horizon Spatial Biologycited by 3
- 2026EpiBench: Verifiable Evaluation of AI Agents on Epigenomics Analysiscited by 1
- 2026TxBench-PP: Analyzing AI Agent Performance on Small-Molecule Preclinical Pharmacologycited by 1
- 2026scBench-Long: Verifiable Benchmarking of Long-Horizon Single-Cell Biologycited by 1
- 2026Evaluating Agentic Bioinformatics through Function, Evidence, and Validationcited by 0
- 2026Evaluating Agentic Bioinformatics through Function, Evidence, and Validation30 Jul 2026
- 2026scBench-Long: Verifiable Benchmarking of Long-Horizon Single-Cell Biology25 Jun 2026
- 2026SP-Mind: An Autonomous Reasoning Agent for Spatial Proteomics Analysis23 Jun 2026
- 2026TxBench-PP: Analyzing AI Agent Performance on Small-Molecule Preclinical Pharmacology17 Jun 2026
- 2026EpiBench: Verifiable Evaluation of AI Agents on Epigenomics Analysis11 Jun 2026
- 2026Verifiable Benchmarking of Long-Horizon Spatial Biology27 May 2026