APEXA: Execution-Integrity Enforcement for Multi-Agent LLM Automation of Synchrotron Data Reduction
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
LLM agents can make expert scientific workflows accessible through natural language, but a plausible response does not establish that the requested computation ran. This matters at synchrotron facilities, where calibration and integration are multi-step operations over terabyte-scale datasets. We present APEXA, a deployed framework coordinating 61 tools for synchrotron data reduction. A deterministic execution-integrity guard prevents it from returning results for tool calls that did not execute, and the same tool layer enforces motor safety: across 50 adversarial scenarios and four models, it produced 0/200 violations against a simulated EPICS controller, versus 15/200 for a prompt-only baseline. We also introduce APEXA-Bench, 58 facility tasks organized by scientific correctness and physical consequence. On real APS beamline data, one prompt recovered detector geometry and integrated a full attenuation and exposure sweep. Trustworthy scientific agents need deterministic validation of execution, not model-side assurances. We release the framework, benchmark, and traces.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 16 of its 41 references that are in the map. First posted on arXiv 21 Sep 2026.
Builds on
16 of 41 references.
References · 16
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
- 2023GAIA: a benchmark for General AI Assistantscited by 1,084
- 2023ChemCrow: Augmenting large-language models with chemistry toolscited by 609
- 2023Identifying the Risks of LM Agents with an LM-Emulated Sandboxcited by 514
- 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agentscited by 406
- 2023MLAgentBench: Evaluating Language Agents on Machine Learning Experimentationcited by 339
- 2024TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Taskscited by 306
- 2024R-Judge: Benchmarking Safety Risk Awareness for LLM Agentscited by 268
4 more
- 2024ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discoverycited by 208
- 2024SciCode: A Research Coding Benchmark Curated by Scientistscited by 172
- 2024VISION: A Modular AI Assistant for Natural Human-Instrument Interaction at Scientific User Facilitiescited by 10
- 2026Toward Full Autonomous Laboratory Instrumentation Control with Large Language Modelscited by 5
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Automating High Energy Physics Data Analysis with LLM-Powered Agents
- 2026Collider-Bench: Benchmarking AI Agents with Particle Physics Analysis Reproduction
- 2026Grounded autonomous research: a fault-tolerant LLM pipeline from corpus to manuscript in frontier computational physics
- 2025EnvTrace: Simulation-Based Semantic Evaluation of LLM Code via Execution Trace Alignment -- Demonstrated at Synchrotron Beamlines
- 2026EAA: Automating materials characterization with vision language model agents
- 2026AutoTuneBench: Trustworthy Measurement for Agent Auto-Tuning of LLM Serving Engines
- 2026Schema-Bound LLM Control of Scientific Instrumentation through Model Context Protocol Skills
- 2026AI Agents Can Already Autonomously Perform Experimental High Energy Physics