# Agents of Discovery

arXiv: https://arxiv.org/abs/2509.08535 · HTML: https://mlsanity.com/paper/2509.08535/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use.
- Builds on 21 of its 53 references that are in the map.
- First posted on arXiv 10 Sep 2025.

## Placed in

- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 21 of 21)

- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [AgentBench: Evaluating LLMs as Agents](https://mlsanity.com/paper/2308.03688/) (2023)
- [Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting](https://mlsanity.com/paper/2310.11324/) (2023)
- [ChemCrow: Augmenting large-language models with chemistry tools](https://mlsanity.com/paper/2304.05376/) (2023)
- [Why do tree-based models still outperform deep learning on tabular data?](https://mlsanity.com/paper/2207.08815/) (2022)
- [A Living Review of Machine Learning for Particle Physics](https://mlsanity.com/paper/2102.02770/) (2021)
- [Anomaly Detection with Density Estimation](https://mlsanity.com/paper/2001.04990/) (2020)
- [SciAgent: Tool-augmented Language Models for Scientific Reasoning](https://mlsanity.com/paper/2402.11451/) (2024)
- [Machine Learning in the Search for New Fundamental Physics](https://mlsanity.com/paper/2112.03769/) (2021)
- [Classification without labels: Learning from mixed samples in high energy physics](https://mlsanity.com/paper/1708.02949/) (2017)
- [Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluation](https://mlsanity.com/paper/2502.05151/) (2025)
- [Machine Learning for Anomaly Detection in Particle Physics](https://mlsanity.com/paper/2312.14190/) (2023)
- [Iterative Translation Refinement with Large Language Models](https://mlsanity.com/paper/2306.03856/) (2023)
- [Large Physics Models: Towards a collaborative approach with Large Language Models and Foundation Models](https://mlsanity.com/paper/2501.05382/) (2025)
- [Simulation-Assisted Decorrelation for Resonant Anomaly Detection](https://mlsanity.com/paper/2009.02205/) (2020)
- [Large Language Models for Human-Machine Collaborative Particle Accelerator Tuning through Natural Language](https://mlsanity.com/paper/2405.08888/) (2024)
- [Multi3Hate: Multimodal, Multilingual, and Multicultural Hate Speech Detection with Vision-Language Models](https://mlsanity.com/paper/2411.03888/) (2024)
- [Around the World in 24 Hours: Probing LLM Knowledge of Time and Place](https://mlsanity.com/paper/2506.03984/) (2025)
- [Decision-Making with Deliberation: Meta-reviewing as a Document-grounded Dialogue](https://mlsanity.com/paper/2508.05283/) (2025)

## Built on by (most cited first, 3 of 3)

- [AI Agents Can Already Autonomously Perform Experimental High Energy Physics](https://mlsanity.com/paper/2603.20179/) (2026)
- [Machine Learning Meets High-Energy Nuclear Physics: From Pattern Recognition to Physics-Integrated Discovery](https://mlsanity.com/paper/2610.12293/) (2026)
- [Beyond a Better Score: Long-Horizon Agentic ML Development and Evaluation Protocol for Physics Time Series](https://mlsanity.com/paper/2610.14024/) (2026)

## Abstract (arXiv)

The substantial data volumes encountered in modern particle physics and other domains of fundamental physics research allow (and require) the use of increasingly complex data analysis tools and workflows. While the use of machine learning (ML) tools for data analysis has recently proliferated, these tools are typically special-purpose algorithms that rely, for example, on encoded physics knowledge to reach optimal performance. In this work, we investigate a new and orthogonal direction: Using recent progress in large language models (LLMs) to create a team of agents -- instances of LLMs with specific subtasks -- that jointly solve data analysis-based research problems in a way similar to how a human researcher might: by creating code to operate standard tools and libraries (including ML systems) and by building on results of previous iterations. If successful, such agent-based systems could be deployed to automate routine analysis components to counteract the increasing complexity of modern tool chains. To investigate the capabilities of current-generation commercial LLMs, we consider the task of anomaly detection via the publicly available and highly-studied LHC Olympics dataset. Several current models by OpenAI (GPT-4o, o4-mini, GPT-4.1, and GPT-5) are investigated and their stability tested. Overall, we observe the capacity of the agent-based system to solve this data analysis problem. The best agent-created solutions mirror the performance of human state-of-the-art results.