# Testing GPT-4 with Wolfram Alpha and Code Interpreter plug-ins on math and science problems

arXiv: https://arxiv.org/abs/2308.05713 · HTML: https://mlsanity.com/paper/2308.05713/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 54 references that are in the map.
- First posted on arXiv 10 Aug 2023.

## Builds on (most cited first, 7 of 7)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering](https://mlsanity.com/paper/1809.02789/) (2018)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering](https://mlsanity.com/paper/2209.09513/) (2022)
- [Mathematical Capabilities of ChatGPT](https://mlsanity.com/paper/2301.13867/) (2023)
- [SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Models](https://mlsanity.com/paper/2307.10635/) (2023)

## Built on by (most cited first, 12 of 12)

- [CLadder: Assessing Causal Reasoning in Language Models](https://mlsanity.com/paper/2312.04350/) (2023)
- [MathOdyssey: Benchmarking Mathematical Problem-Solving Skills in Large Language Models Using Odyssey Math Data](https://mlsanity.com/paper/2406.18321/) (2024)
- [ChemAgent: Self-updating Library in Large Language Models Improves Chemical Reasoning](https://mlsanity.com/paper/2501.06590/) (2025)
- [A Philosophical Introduction to Language Models - Part II: The Way Forward](https://mlsanity.com/paper/2405.03207/) (2024)
- [AI for Mathematics: A Cognitive Science Perspective](https://mlsanity.com/paper/2310.13021/) (2023)
- [Can Theoretical Physics Research Benefit from Language Agents?](https://mlsanity.com/paper/2506.06214/) (2025)
- [Relational Programming with Foundation Models](https://mlsanity.com/paper/2412.14515/) (2024)
- [Evaluation of LLMs for mathematical problem solving](https://mlsanity.com/paper/2506.00309/) (2025)
- [Conceptual Model Interpreter for Large Language Models](https://mlsanity.com/paper/2311.07605/) (2023)
- [ChatGPT-4 with Code Interpreter can be used to solve introductory college-level vector calculus and electromagnetism problems](https://mlsanity.com/paper/2309.08881/) (2023)
- [New Horizons: Pioneering Pharmaceutical R&D with Generative AI from lab to the clinic -- an industry perspective](https://mlsanity.com/paper/2312.12482/) (2023)
- [Masked Diffusion Language Agents for Tool-Integrated Chemical Reasoning](https://mlsanity.com/paper/2610.13330/) (2026)

## Abstract (arXiv)

This report describes a test of the large language model GPT-4 with the Wolfram Alpha and the Code Interpreter plug-ins on 105 original problems in science and math, at the high school and college levels, carried out in June-August 2023. Our tests suggest that the plug-ins significantly enhance GPT's ability to solve these problems. Having said that, there are still often "interface" failures; that is, GPT often has trouble formulating problems in a way that elicits useful answers from the plug-ins. Fixing these interface failures seems like a central challenge in making GPT a reliable tool for college-level calculation problems.