mlsanityData to 14 Oct 2026

Can Theoretical Physics Research Benefit from Language Agents?

arXiv 2506.06214 ↗First posted 6 Jun 2025cs.CL cs.AI math-phcited by 9

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

Large Language Models (LLMs) are rapidly advancing across diverse domains, yet their application in theoretical physics remains inadequate. While current models show competence in mathematical reasoning and code generation, we identify critical gaps in physical intuition, constraint satisfaction, and reliable reasoning that cannot be addressed through prompting alone. Physics demands approximation judgment, symmetry exploitation, and physical grounding that require AI agents specifically trained on physics reasoning patterns and equipped with physics-aware verification tools. We argue that LLM would require such domain-specialized training and tooling to be useful in real-world for physics research. We envision physics-specialized AI agents that seamlessly handle multimodal data, propose physically consistent hypotheses, and autonomously verify theoretical results. Realizing this vision requires developing physics-specific training datasets, reward signals that capture physical reasoning quality, and verification frameworks encoding fundamental principles. We call for collaborative efforts between physics and AI communities to build the specialized infrastructure necessary for AI-driven scientific discovery.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 58 of its 136 references that are in the map. First posted on arXiv 6 Jun 2025.

Builds on

58 of 136 references.

References · 58

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  6. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  7. 2023Let's Verify Step by Stepcited by 3,789
  8. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  9. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  10. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  11. 2020Longformer: The Long-Document Transformercited by 3,413
  12. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
38 more
  1. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  2. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  3. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
  4. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  5. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,414
  6. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,392
  7. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
  8. 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
  9. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
  10. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,190
  11. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,071
  12. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,055
  13. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 894
  14. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 886
  15. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 841
  16. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 698
  17. 2022PAL: Program-aided Language Modelscited by 652
  18. 2023ChemCrow: Augmenting large-language models with chemistry toolscited by 631
  19. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 574
  20. 2023Are Emergent Abilities of Large Language Models a Mirage?cited by 523
  21. 2023Cognitive Architectures for Language Agentscited by 429
  22. 2023Llemma: An Open Language Model For Mathematicscited by 388
  23. 2022Self-critiquing models for assisting human evaluatorscited by 380
  24. 2025PaperBench: Evaluating AI's Ability to Replicate AI Researchcited by 321
  25. 2023SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Modelscited by 256
  26. 2023Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Taskscited by 246
  27. 2024FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AIcited by 243
  28. 2024ReFT: Reasoning with Reinforced Fine-Tuningcited by 239
  29. 2023How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language modelcited by 202
  30. 2024Foundational Challenges in Assuring Alignment and Safety of Large Language Modelscited by 199
  31. 2022Language Models of Code are Few-Shot Commonsense Learnerscited by 160
  32. 2024LLM Critics Help Catch LLM Bugscited by 147
  33. 2023SciEval: A Multi-Level Large Language Model Evaluation Benchmark for Scientific Researchcited by 115
  34. 2025MATH-Perturb: Benchmarking LLMs' Math Reasoning Abilities against Hard Perturbationscited by 89
  35. 2024Prover-Verifier Games improve legibility of LLM outputscited by 85
  36. 2025PHYBench: Holistic Evaluation of Physical Perception and Reasoning in Large Language Modelscited by 61
  37. 2025PhysReason: A Comprehensive Benchmark towards Physics-Based Reasoningcited by 59
  38. 2023Managing extreme AI risks amid rapid progresscited by 48

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 6.

9 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026TheoPhysEval: Assessing AI agents at autonomous theoretical physics problem solving
  2. 2025Advancing AI-Scientist Understanding: Multi-Agent LLMs with Interpretable Physics Reasoning
  3. 2025PhysicsEval: Inference-Time Techniques to Improve the Reasoning Proficiency of Large Language Models on Physics Problems
  4. 2025Large Physics Models: Towards a collaborative approach with Large Language Models and Foundation Models
  5. 2025PhysMaster: Building an Autonomous AI Physicist for Theoretical and Computational Physics Research
  6. 2026PhysMent: An Interactive Approach For LLM Reasoning In Physics Problems
  7. 2025PhySense: Principle-Based Physics Reasoning Benchmarking for Large Language Models
  8. 2026PRL-Bench: A Comprehensive Benchmark Evaluating LLMs' Capabilities in Frontier Physics Research