mlsanityData to 7 Oct 2026

PRL-Bench: A Comprehensive Benchmark Evaluating LLMs' Capabilities in Frontier Physics Research

arXiv 2604.15411 ↗First posted 16 Apr 2026cs.LG cs.AI physics.data-ancited by 5

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

The paradigm of agentic science requires AI systems to conduct robust reasoning and engage in long-horizon, autonomous exploration. However, current scientific benchmarks remain confined to domain knowledge comprehension and complex reasoning, failing to evaluate the exploratory nature and procedural complexity of real-world research. In this work, we present research-oriented evaluations in theoretical and computational physics, a natural testbed with comprehensive domain knowledge, complex reasoning, and verifiable end-to-end workflows without reliance on experiments. Here we introduce PRL-Bench (Physics Research by LLMs), a benchmark designed to systematically map the capability boundaries of LLMs in executing end-to-end physics research. Constructed from 100 curated papers from the latest issues of Physical Review Letters since August 2025 and validated by domain experts, PRL-Bench covers five major theory- and computation-intensive subfields of modern physics: astrophysics, condensed matter physics, high-energy physics, quantum information, and statistical physics. Each task in the benchmark is designed to replicate the core properties of authentic scientific research, including exploration-oriented formulation, long-horizon workflows, and objective verifiability, thereby reconstructing the essential reasoning processes and research workflows of real physics research. Evaluation across frontier models shows that performance remains limited, with the best overall score below 50, revealing a pronounced gap between current LLM capabilities and the demands of real scientific research. PRL-Bench serves a reliable testbed for accessing next generation AI scientists advancing AI systems toward autonomous scientific discovery.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 12 of its 19 references that are in the map. First posted on arXiv 16 Apr 2026.

Builds on

12 of 19 references.

Closest ancestors · 1

Same direction, newest first

  1. 2026GRACE: an Agentic AI for Particle Physics Experiment Design and Simulation31 Jan

Other references · 11

  1. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
  2. 2025Accelerating scientific discovery with Co-Scientistcited by 353
  3. 2023SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Modelscited by 246
  4. 2025Kosmos: An AI Scientist for Autonomous Discoverycited by 78
  5. 2025Challenging the Boundaries of Reasoning: An Olympiad-Level Math Benchmark for Large Language Modelscited by 65
  6. 2025PHYBench: Holistic Evaluation of Physical Perception and Reasoning in Large Language Modelscited by 57
  7. 2025Robin: A multi-agent system for automating scientific discoverycited by 57
  8. 2026FrontierScience: Evaluating AI's Ability to Perform Expert-Level Scientific Taskscited by 42
  9. 2025Theoretical Physics Benchmark (TPBench) -- a Dataset and Study of AI Reasoning Capabilities in Theoretical Physicscited by 33
  10. 2025PHYSICS: Benchmarking Foundation Models on University-Level Physics Problem Solvingcited by 31
  11. 2026An End-to-end Architecture for Collider Physics and Beyondcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025PhysMaster: Building an Autonomous AI Physicist for Theoretical and Computational Physics Research
  2. 2025PhysGym: Benchmarking LLMs in Interactive Physics Discovery with Controlled Priors
  3. 2026DiscoverPhysics: Benchmarking LLMs for Out-of-the-Box Scientific Thinking
  4. 2026Grounded autonomous research: a fault-tolerant LLM pipeline from corpus to manuscript in frontier computational physics
  5. 2025Can Theoretical Physics Research Benefit from Language Agents?
  6. 2025NewtonBench: Benchmarking Generalizable Scientific Law Discovery in LLM Agents
  7. 2025PhysicsEval: Inference-Time Techniques to Improve the Reasoning Proficiency of Large Language Models on Physics Problems
  8. 2025Large Physics Models: Towards a collaborative approach with Large Language Models and Foundation Models