mlsanityData to 7 Oct 2026

Testing GPT-4 with Wolfram Alpha and Code Interpreter plug-ins on math and science problems

arXiv 2308.05713 ↗First posted 10 Aug 2023cs.AI math.HO physics.pop-phcited by 11

Not on the map. Why?

Abstract (arXiv)

This report describes a test of the large language model GPT-4 with the Wolfram Alpha and the Code Interpreter plug-ins on 105 original problems in science and math, at the high school and college levels, carried out in June-August 2023. Our tests suggest that the plug-ins significantly enhance GPT's ability to solve these problems. Having said that, there are still often "interface" failures; that is, GPT often has trouble formulating problems in a way that elicits useful answers from the plug-ins. Fixing these interface failures seems like a central challenge in making GPT a reliable tool for college-level calculation problems.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 7 of its 54 references that are in the map. First posted on arXiv 10 Aug 2023.

Builds on

7 of 54 references.

References · 7

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  3. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  4. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  5. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  6. 2023Mathematical Capabilities of ChatGPTcited by 263
  7. 2023SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Modelscited by 246

Built on by

Cited by year

Citing papers per year: 2023: 5; 2024: 3; 2025: 3.

11 papers.

  1. 2023CLadder: Assessing Causal Reasoning in Language Modelscited by 102
  2. 2024MathOdyssey: Benchmarking Mathematical Problem-Solving Skills in Large Language Models Using Odyssey Math Datacited by 51
  3. 2025ChemAgent: Self-updating Library in Large Language Models Improves Chemical Reasoningcited by 41
  4. 2024A Philosophical Introduction to Language Models - Part II: The Way Forwardcited by 17
  5. 2023AI for Mathematics: A Cognitive Science Perspectivecited by 10
  6. 2025Can Theoretical Physics Research Benefit from Language Agents?cited by 9
  7. 2024Relational Programming with Foundation Modelscited by 7
  8. 2025Evaluation of LLMs for mathematical problem solvingcited by 7
  9. 2023Conceptual Model Interpreter for Large Language Modelscited by 2
  10. 2023ChatGPT-4 with Code Interpreter can be used to solve introductory college-level vector calculus and electromagnetism problemscited by 1
  11. 2023New Horizons: Pioneering Pharmaceutical R&D with Generative AI from lab to the clinic -- an industry perspectivecited by 0