Glite ARF: Verifier-Driven Research with Parallel LLM Coding Agents
On the map: LLM agents and tool use → Web, search, and research agents
Abstract (arXiv)
LLM coding agents make it tempting to automate empirical research by delegating experiments to them directly, but naive delegation does not scale to large projects: low-rate instruction lapses compound into broken, irreproducible artefacts. To address this problem, we present Glite ARF, an open-source Python framework for running many LLM coding agents in parallel on a research repository without sacrificing reproducibility or auditability. The framework defines a three-role stack: a human researcher chooses which hypotheses to test, coding agents (Claude Code, Codex CLI) implement individual tasks under a fixed structure, and deterministic Python verifier scripts enforce task isolation, immutability of completed work, a corrections overlay, and a materialised project overview. We call this verifier-driven research: the rules of the research process live in code that fails loudly when violated, not in prose that agents are merely asked to follow. Using Glite ARF, we developed our submission to the BEA 2026 vocabulary-difficulty shared task, placing first in the closed track and second in the open track on all three target languages (Spanish, German, Mandarin) and reducing the official baseline RMSE by 29.9% (closed) and 35.9% (open). The campaign comprised 273 tracked tasks (146 experiment runs) across 129 feature sets, run by up to twelve parallel agents orchestrated from a single laptop - with some model training on rented A100s - at approximately 450 in LLM API spend (498 total third-party cost), and structured per-fold provenance let us catch and strip four target-leaking feature sets, correcting an implausible 0.609 RMSE to 0.802. Across three campaigns in three domains, the framework's structural machinery adds only about 1% of wall-clock time. Framework and a public demo project accompany this paper.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 12 of its 32 references that are in the map. First posted on arXiv 25 Jun 2026.
Builds on
12 of 32 references.
Closest ancestors · 4
Same direction, newest first
- 2025The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search10 Apr
- 2025Agent Laboratory: Using LLM Agents as Research Assistants8 Jan
- 2024MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering9 Oct
- 2023MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation5 Oct
Other references · 8
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
- 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
- 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,002
- 2025Measuring AI Ability to Complete Long Software Taskscited by 142
- 2026CORAL: Towards Autonomous Multi-Agent Evolution for Open-Ended Discoverycited by 38
- 2025Evaluating Sakana's AI Scientist: Bold Claims, Mixed Results, and a Promising Future?cited by 32
- 2025Ace-CEFR -- A Dataset for Automated Evaluation of the Linguistic Difficulty of Conversational Texts for LLM Applicationscited by 4
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026CURATE: Leveraging LLM Agents to Compose, Catalog, and Deploy Reproducible Workflows
- 2025From Reproduction to Replication: Evaluating Research Agents with Progressive Code Masking
- 2026A Deterministic Control Plane for LLM Coding Agents
- 2026Can Coding Agents Reproduce Findings in Computational Materials Science?
- 2025RECODE-H: A Benchmark for Research Code Development with Interactive Human Feedback
- 2026Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap
- 2026AutoLLMResearch: Training Research Agents for Automating LLM Experiment Configuration - Learning from Cheap, Optimizing Expensive
- 2025MOSAIC: Multi-agent Orchestration for Task-Intelligent Scientific Coding