mlsanityData to 14 Oct 2026

Solving Linear Algebra by Program Synthesis

arXiv 2111.08171 ↗First posted 16 Nov 2021cs.LG cs.AI cs.CLcited by 14

Not on the map. Why?

Abstract (arXiv)

We solve MIT's Linear Algebra 18.06 course and Columbia University's Computational Linear Algebra COMS3251 courses with perfect accuracy by interactive program synthesis. This surprisingly strong result is achieved by turning the course questions into programming tasks and then running the programs to produce the correct answers. We use OpenAI Codex with zero-shot learning, without providing any examples in the prompts, to synthesize code from questions. We quantify the difference between the original question text and the transformed question text that yields a correct answer. Since all COMS3251 questions are not available online the model is not overfitting. We go beyond just generating code for questions with numerical answers by interactively generating code that also results visually pleasing plots as output. Finally, we automatically generate new questions given a few sample questions which may be used as new course content. This work is a significant step forward in solving quantitative math problems and opens the door for solving many university level STEM courses by machine.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 10 of its 16 references that are in the map. First posted on arXiv 16 Nov 2021.

Builds on

10 of 16 references.

References · 10

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  5. 2021MathBERT: A Pre-Trained Model for Mathematical Formula Understandingcited by 58
  6. 2021MWPToolkit: An Open-Source Framework for Deep Learning-Based Math Word Problem Solverscited by 39
  7. 2020Semantically-Aligned Universal Tree-Structured Solver for Math Word Problemscited by 35
  8. 2020Graph-to-Tree Neural Networks for Learning Structured Input-Output Translation with Applications to Semantic Parsing and Math Word Problemcited by 31
  9. 2021Sequence to General Tree: Knowledge-Guided Geometry Word Problem Solvingcited by 11
  10. 2021Solving Machine Learning Problemscited by 4

Built on by

Cited by year

Citing papers per year: 2022: 8; 2023: 6.

14 papers.

Sort:

By area:

  1. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  2. 2022GitHub Copilot AI pair programmer: Asset or Liability?cited by 152
  3. 2022Large Language Models Meet NL2Code: A Surveycited by 138
  4. 2023Unifying the Perspectives of NLP and Software Engineering: A Survey on Language Models for Codecited by 62
  5. 2023Thrilled by Your Progress! Large Language Models (GPT-4) No Longer Struggle to Pass Assessments in Higher Education Programming Coursescited by 39
  6. 2023Large Language Models (GPT) Struggle to Answer Multiple-Choice Questions about Codecited by 32
  7. 2023Can GPT-4 Support Analysis of Textual Data in Tasks Requiring Highly Specialized Domain Expertise?cited by 31
  8. 2023Exploring the Robustness of Large Language Models for Solving Programming Problemscited by 16
  9. 2022Codex Hacks HackerRank: Memorization Issues and a Framework for Code Synthesis Evaluationcited by 14
  10. 2022A Survey on Pretrained Language Models for Neural Code Intelligencecited by 13
  11. 2022Fooling MOSS Detection with Pretrained Language Modelscited by 12
  12. 2022Learning to Configure Computer Networks with Neural Algorithmic Reasoningcited by 5
2 more
  1. 2022From Natural Language to Simulations: Applying GPT-3 Codex to Automate Simulation Modeling of Logistics Systemscited by 0
  2. 2023Can Generative Pre-trained Transformers (GPT) Pass Assessments in Higher Education Programming Courses?cited by 0