mlsanityData to 7 Oct 2026

SciAgent: Tool-augmented Language Models for Scientific Reasoning

arXiv 2402.11451 ↗First posted 18 Feb 2024cs.CL cs.AIcited by 69

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

also in 1 more
Abstract (arXiv)

Scientific reasoning poses an excessive challenge for even the most advanced Large Language Models (LLMs). To make this task more practical and solvable for LLMs, we introduce a new task setting named tool-augmented scientific reasoning. This setting supplements LLMs with scalable toolsets, and shifts the focus from pursuing an omniscient problem solver to a proficient tool-user. To facilitate the research of such setting, we construct a tool-augmented training corpus named MathFunc which encompasses over 30,000 samples and roughly 6,000 tools. Building on MathFunc, we develop SciAgent to retrieve, understand and, if necessary, use tools for scientific problem solving. Additionally, we craft a benchmark, SciToolBench, spanning five scientific domains to evaluate LLMs' abilities with tool assistance. Extensive experiments on SciToolBench confirm the effectiveness of SciAgent. Notably, SciAgent-Mistral-7B surpasses other LLMs with the same size by more than 13% in absolute accuracy. Furthermore, SciAgent-DeepMath-7B shows much superior performance than ChatGPT.

Key facts. Cited by 69 papers in the map (data through 7 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 36 of its 72 references that are in the map. First posted on arXiv 18 Feb 2024.

Builds on

36 of 72 references.

References · 36

  1. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  7. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  8. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  9. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  10. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  11. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  12. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
24 more
  1. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,449
  2. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  3. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  4. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  5. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  6. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 751
  7. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 694
  8. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 618
  9. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 483
  10. 2023Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoningcited by 379
  11. 2023Large Language Models as Tool Makerscited by 293
  12. 2023MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedbackcited by 293
  13. 2021ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learningcited by 253
  14. 2023GPT4Tools: Teaching Large Language Model to Use Tools via Self-instructioncited by 245
  15. 2023ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddingscited by 238
  16. 2023TheoremQA: A Theorem-driven Question Answering datasetcited by 189
  17. 2023Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verificationcited by 166
  18. 2023MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoningcited by 155
  19. 2023RestGPT: Connecting Large Language Models with Real-World RESTful APIscited by 129
  20. 2023CRAFT: Customizing LLMs by Creating and Retrieving from Specialized Toolsetscited by 122
  21. 2023GeneGPT: Augmenting Large Language Models with Domain Tools for Improved Access to Biomedical Informationcited by 115
  22. 2023SciEval: A Multi-Level Large Language Model Evaluation Benchmark for Scientific Researchcited by 112
  23. 2024TroVE: Inducing Verifiable and Efficient Toolboxes for Solving Programmatic Taskscited by 70
  24. 2023Confucius: Iterative Tool Learning from Introspection Feedback by Easy-to-Difficult Curriculumcited by 64

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 13; 2025: 28; 2026: 28.

69 papers.

Sort:

By area:

  1. 2024Tool Learning with Large Language Models: A Surveycited by 221
  2. 2025AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challengescited by 167
  3. 2024RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human expertscited by 153
  4. 2025A Survey of Context Engineering for Large Language Modelscited by 120
  5. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 108
  6. 2024A Review of Large Language Models and Autonomous Agents in Chemistrycited by 96
  7. 2025Towards Scientific Intelligence: A Survey of LLM-based Scientific Agentscited by 96
  8. 2024CriticBench: Benchmarking LLMs for Critique-Correct Reasoningcited by 88
  9. 2025From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discoverycited by 74
  10. 2025LLM Agents for Education: Advances and Applicationscited by 59
  11. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 47
  12. 2025UGPhysics: A Comprehensive Benchmark for Undergraduate Physics Reasoning with Large Language Modelscited by 45
38 more
  1. 2025A Survey on the Optimization of Large Language Model-based Agentscited by 45
  2. 2025Towards Agentic RAG with Deep Reasoning: A Survey of RAG-Reasoning Systems in LLMscited by 40
  3. 2025LMR-BENCH: Evaluating LLM Agent's Ability on Reproducing Language Modeling Researchcited by 29
  4. 2024Planning with Multi-Constraints via Collaborative Language Agentscited by 26
  5. 2025Toward Generalizable Evaluation in the LLM Era: A Survey Beyond Benchmarkscited by 26
  6. 2024Security of AI Agentscited by 24
  7. 2024Improving Physics Reasoning in Large Language Models Using Mixture of Refinement Agentscited by 22
  8. 2026ReCreate: Reasoning and Creating Domain Agents Driven by Experiencecited by 22
  9. 2024The Fusion of Large Language Models and Formal Methods for Trustworthy AI Agents: A Roadmapcited by 20
  10. 2026Large Language Model Reasoning Failurescited by 20
  11. 2024Husky: A Unified, Open-Source Language Agent for Multi-Step Reasoningcited by 16
  12. 2025Empowering Scientific Workflows with Federated Agentscited by 16
  13. 2025MatSciBench: Benchmarking the Reasoning Ability of Large Language Models in Materials Sciencecited by 15
  14. 2025StableToolBench-MirrorAPI: Modeling Tool Environments as Mirrors of 7,000+ Real-World APIscited by 14
  15. 2025Generalizable End-to-End Tool-Use RL with Synthetic CodeGymcited by 14
  16. 2025Standard Benchmarks Fail -- Auditing LLM Agents in Finance Must Prioritize Riskcited by 13
  17. 2026SciAgentGym: Benchmarking Multi-Step Scientific Tool-use in LLM Agentscited by 12
  18. 2026Toward Efficient Agents: Memory, Tool learning, and Planningcited by 11
  19. 2024TART: An Open-Source Tool-Augmented Framework for Explainable Table-based Reasoningcited by 10
  20. 202534 Examples of LLM Applications in Materials Science and Chemistry: Towards Automation, Assistants, Agents, and Accelerated Scientific Discoverycited by 9
  21. 2024Reflections from the 2024 Large Language Model (LLM) Hackathon for Applications in Materials Science and Chemistrycited by 8
  22. 2026Memory Intelligence Agentcited by 6
  23. 2025FamilyTool: A Multi-hop Personalized Tool Use Benchmarkcited by 5
  24. 2025Conversational Education at Scale: A Multi-LLM Agent Workflow for Procedural Learning and Pedagogic Quality Assessmentcited by 5
  25. 2025ToolLibGen: Scalable Automatic Tool Creation and Aggregation for LLM Reasoningcited by 5
  26. 2026Investigating Tool-Memory Conflicts in Tool-Augmented LLMscited by 5
  27. 2024NesTools: A Dataset for Evaluating Nested Tool Learning Abilities of Large Language Modelscited by 4
  28. 2025MEENA (PersianMMMU): Multimodal-Multilingual Educational Exams for N-level Assessmentcited by 3
  29. 2025SciEvalKit: An Open-source Evaluation Toolkit for Scientific General Intelligencecited by 3
  30. 2025An Auditable Agent Platform For Automated Molecular Optimisationcited by 2
  31. 2025Agents of Discoverycited by 2
  32. 2025OpenLens AI: Fully Autonomous Research Agent for Health Infomaticscited by 2
  33. 2026STRIDE: A Self-Reflective Agent Framework for Reliable Automatic Equation Discoverycited by 2
  34. 2026Harnessing Generalist Agents for Contextualized Time Seriescited by 2
  35. 2025RAISE: Enhancing Scientific Reasoning in LLMs via Step-by-Step Retrievalcited by 1
  36. 2026STEMVerse: A Dual-Axis Diagnostic Framework for STEM Reasoning in Large Language Modelscited by 1
  37. 2026Enhancing Tool Calling in LLMs with the International Tool Calling Datasetcited by 1
  38. 2026SinkTrack: Attention Sink based Context Anchoring for Large Language Modelscited by 1

50 most cited of 69 All 69 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Models
  2. 2024ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discovery
  3. 2025SciAgent: A Unified Multi-Agent System for Generalistic Scientific Reasoning
  4. 2026SciVQR: A Multidisciplinary Multimodal Benchmark for Advanced Scientific Reasoning Evaluation
  5. 2025Demystifying Scientific Problem-Solving in LLMs by Probing Knowledge and Reasoning
  6. 2025CURIE: Evaluating LLMs On Multitask Scientific Long Context Understanding and Reasoning
  7. 2025PRiSM: An Agentic Multimodal Benchmark for Scientific Reasoning via Python-Grounded Evaluation
  8. 2025SCI-Verifier: Scientific Verifier with Thinking