# Solving Linear Algebra by Program Synthesis

arXiv: https://arxiv.org/abs/2111.08171 · HTML: https://mlsanity.com/paper/2111.08171/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 16 references that are in the map.
- First posted on arXiv 16 Nov 2021.

## Builds on (most cited first, 10 of 10)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [MathBERT: A Pre-Trained Model for Mathematical Formula Understanding](https://mlsanity.com/paper/2105.00377/) (2021)
- [MWPToolkit: An Open-Source Framework for Deep Learning-Based Math Word Problem Solvers](https://mlsanity.com/paper/2109.00799/) (2021)
- [Semantically-Aligned Universal Tree-Structured Solver for Math Word Problems](https://mlsanity.com/paper/2010.06823/) (2020)
- [Graph-to-Tree Neural Networks for Learning Structured Input-Output Translation with Applications to Semantic Parsing and Math Word Problem](https://mlsanity.com/paper/2004.13781/) (2020)
- [Sequence to General Tree: Knowledge-Guided Geometry Word Problem Solving](https://mlsanity.com/paper/2106.00990/) (2021)
- [Solving Machine Learning Problems](https://mlsanity.com/paper/2107.01238/) (2021)

## Built on by (most cited first, 14 of 14)

- [Competition-Level Code Generation with AlphaCode](https://mlsanity.com/paper/2203.07814/) (2022)
- [GitHub Copilot AI pair programmer: Asset or Liability?](https://mlsanity.com/paper/2206.15331/) (2022)
- [Large Language Models Meet NL2Code: A Survey](https://mlsanity.com/paper/2212.09420/) (2022)
- [Unifying the Perspectives of NLP and Software Engineering: A Survey on Language Models for Code](https://mlsanity.com/paper/2311.07989/) (2023)
- [Thrilled by Your Progress! Large Language Models (GPT-4) No Longer Struggle to Pass Assessments in Higher Education Programming Courses](https://mlsanity.com/paper/2306.10073/) (2023)
- [Large Language Models (GPT) Struggle to Answer Multiple-Choice Questions about Code](https://mlsanity.com/paper/2303.08033/) (2023)
- [Can GPT-4 Support Analysis of Textual Data in Tasks Requiring Highly Specialized Domain Expertise?](https://mlsanity.com/paper/2306.13906/) (2023)
- [Exploring the Robustness of Large Language Models for Solving Programming Problems](https://mlsanity.com/paper/2306.14583/) (2023)
- [Codex Hacks HackerRank: Memorization Issues and a Framework for Code Synthesis Evaluation](https://mlsanity.com/paper/2212.02684/) (2022)
- [A Survey on Pretrained Language Models for Neural Code Intelligence](https://mlsanity.com/paper/2212.10079/) (2022)
- [Fooling MOSS Detection with Pretrained Language Models](https://mlsanity.com/paper/2201.07406/) (2022)
- [Learning to Configure Computer Networks with Neural Algorithmic Reasoning](https://mlsanity.com/paper/2211.01980/) (2022)
- [From Natural Language to Simulations: Applying GPT-3 Codex to Automate Simulation Modeling of Logistics Systems](https://mlsanity.com/paper/2202.12107/) (2022)
- [Can Generative Pre-trained Transformers (GPT) Pass Assessments in Higher Education Programming Courses?](https://mlsanity.com/paper/2303.09325/) (2023)

## Abstract (arXiv)

We solve MIT's Linear Algebra 18.06 course and Columbia University's Computational Linear Algebra COMS3251 courses with perfect accuracy by interactive program synthesis. This surprisingly strong result is achieved by turning the course questions into programming tasks and then running the programs to produce the correct answers. We use OpenAI Codex with zero-shot learning, without providing any examples in the prompts, to synthesize code from questions. We quantify the difference between the original question text and the transformed question text that yields a correct answer. Since all COMS3251 questions are not available online the model is not overfitting. We go beyond just generating code for questions with numerical answers by interactively generating code that also results visually pleasing plots as output. Finally, we automatically generate new questions given a few sample questions which may be used as new course content. This work is a significant step forward in solving quantitative math problems and opens the door for solving many university level STEM courses by machine.