Building Agent Harnesses for Scientific Curation from Multimodal Sources
Not on the map. Why?
Abstract (arXiv)
Scientific discovery workflows often depend on structured curation from the literature. This is difficult for current agents because the key evidence is scattered across long text, dense tables, and figures, and the final records often require reasoning across multiple evidence fragments rather than copying a single span. We study scientific curation from multimodal sources and introduce Beaver, an agent harness that extracts structured information from scientific papers while preserving provenance to the supporting evidence. Beaver combines a frontier agent with multimodal evidence tooling, task scaffolding, and artifact-grounded autoresearch. These components turn curation into a staged, auditable workflow and enable an iterative evaluate--diagnose--revise loop, where persistent run artifacts expose stage-localized failures and guide harness updates. Experiments show that Beaver reaches 81.0 on Gold-Referenced Attribute Score (GRAS), an attribute-level measure of agreement with gold curated records, outperforming frontier agents by over 23 absolute points. Ablations show that task scaffolding, multimodal evidence tooling, and provenance traces each contribute meaningfully to performance, while attribute-level analysis shows the largest gains on high-value attributes that require cross-modal reasoning and normalization. These results show that, for scientific curation from papers with multimodal evidence, harness design is a central determinant of agent performance.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 31 of its 48 references that are in the map. First posted on arXiv 19 Jun 2026.
Builds on
31 of 48 references.
References · 31
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
- 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 803
- 2020Fact or Fiction: Verifying Scientific Claimscited by 546
- 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 515
- 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 475
- 2025GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learningcited by 449
- 2018Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Constructioncited by 321
- 2026Meta-Harness: End-to-End Optimization of Model Harnessescited by 248
- 2019Image-based table recognition: data, model, and evaluationcited by 233
- 2021PubTables-1M: Towards comprehensive table extraction from unstructured documentscited by 122
- 2020SciREX: A Challenge Dataset for Document-Level Information Extractioncited by 111
- 2019Complicated Table Structure Recognitioncited by 91
19 more
- 2022BioRED: A Rich Biomedical Relation Extraction Datasetcited by 84
- 2018A Corpus with Multi-Level Annotations of Patients, Interventions and Outcomes to Support Language Processing for Medical Literaturecited by 79
- 2020AxCell: Automatic Extraction of Results from Machine Learning Paperscited by 48
- 2025Feedback Descent: Open-Ended Text Optimization via Pairwise Comparisoncited by 42
- 2022GriTS: Grid table similarity metric for table structure recognitioncited by 27
- 2023Schema-Driven Information Extraction from Heterogeneous Tablescited by 24
- 2024SciER: An Entity and Relation Extraction Dataset for Datasets, Methods, and Tasks in Scientific Documentscited by 20
- 2021TabLeX: A Benchmark Dataset for Structure and Content Information Extraction from Scientific Tablescited by 15
- 2025MetaScale: Test-Time Scaling with Evolving Meta-Thoughtscited by 13
- 2025A foundation model for human-AI collaboration in medical literature miningcited by 11
- 2024cPAPERS: A Dataset of Situated and Multimodal Interactive Conversations in Scientific Paperscited by 10
- 2025Can Multimodal Foundation Models Understand Schematic Diagrams? An Empirical Study on Information-Seeking QA over Scientific Paperscited by 9
- 2024MatViX: Multimodal Information Extraction from Visually Rich Articlescited by 8
- 2026ExtractBench: A Benchmark and Evaluation Methodology for Complex Structured Extractioncited by 8
- 2024Toward Reliable Ad-hoc Scientific Information Extraction: A Case Study on Two Materials Datasetscited by 5
- 2025Intent-Aware Schema Generation And Refinement For Literature Review Tablescited by 5
- 2025SciNLP: A Domain-Specific Benchmark for Full-Text Scientific Entity and Relation Extraction in NLPcited by 4
- 2023CTE: A Dataset for Contextualized Table Extractioncited by 3
- 2024SciDaSynth: Interactive Structured Data Extraction from Scientific Literature with Large Language Modelcited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.
- 2026SRHarness: A Harness for Agentic Symbolic Regressioncited by 0
- 2026SRHarness: A Harness for Agentic Symbolic Regression28 Sep 2026