# Neurodata Without Boredom: Benchmarking Agentic AI for Data Reuse

arXiv: https://arxiv.org/abs/2605.12808 · HTML: https://mlsanity.com/paper/2605.12808/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use.
- Builds on 11 of its 37 references that are in the map.
- First posted on arXiv 12 May 2026.

## Placed in

- LLM agents and tool use / [Medical and scientific domain agents](https://mlsanity.com/agents/medical-and-scientific-domain-agents/)

## Builds on (most cited first, 11 of 11)

- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers](https://mlsanity.com/paper/2105.03011/) (2021)
- [MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation](https://mlsanity.com/paper/2310.03302/) (2023)
- [PaperBench: Evaluating AI's Ability to Replicate AI Research](https://mlsanity.com/paper/2504.01848/) (2025)
- [CORE-Bench: Fostering the Credibility of Published Research Through a Computational Reproducibility Agent Benchmark](https://mlsanity.com/paper/2409.11363/) (2024)
- [A Unified, Scalable Framework for Neural Population Decoding](https://mlsanity.com/paper/2310.16046/) (2023)
- [Human-In-the-Loop Software Development Agents](https://mlsanity.com/paper/2411.12924/) (2024)
- [Towards a "universal translator" for neural dynamics at single-cell, single-spike resolution](https://mlsanity.com/paper/2407.14668/) (2024)
- [Sharing Neurophysiology Data from the Allen Brain Observatory: Lessons Learned](https://mlsanity.com/paper/2212.08638/) (2022)
- [Coarse--graining, fixed points, and scaling in a large population of neurons](https://mlsanity.com/paper/1809.08461/) (2018)
- [A perspective on neuroscience data standardization with Neurodata Without Borders](https://mlsanity.com/paper/2310.04317/) (2023)

## Built on by (most cited first, 1 of 1)

- [A case study of evaluating AI agents on a neuroscience data-to-discovery pipeline](https://mlsanity.com/paper/2606.07718/) (2026)

## Abstract (arXiv)

Neuroscience data are fragmented across labs, experimental paradigms, and storage formats, and data reuse often requires substantial manual effort. Standardized formats have been proposed in response, but a fundamental tension persists: formats flexible enough for diverse experiments are rarely self-explanatory, while fully descriptive datasets demand documentation and curation effort that few labs can sustain. Coding agents, which can read code and text quickly with apparent attention to details humans tend to skim, are a natural candidate to reduce this burden. To measure how well coding agents perform data reuse, we benchmark them on eight recent papers with large-scale mouse neural recordings spanning diverse recording modalities, behavioral paradigms, and formats. Given the data, code, and paper, the agents must reformat the data for a common downstream task: training a decoder from neural activity to task and behavioral variables. Evaluating the resulting solutions, however, is challenging, since many conversion decisions are under-specified and several choices are defensible. Yet a subtle but scientifically significant error can invalidate the dataset in ways aggregate metrics can miss. We therefore pair outcome-based metrics with a process-based rubric, scored by two human evaluators against manual reference solutions. We find that agents succeed on most subtasks but rarely produce a fully error-free end-to-end solution; we characterize their mistakes and suggest data-sharing practices for agent-assisted reuse. Finally, we find that agent judges are unreliable at catching errors on their own, but can automate process-based evaluation when given reference solutions.