# Generative AI performance in core undergraduate mathematics: a curriculum-level case study

arXiv: https://arxiv.org/abs/2509.13359 · HTML: https://mlsanity.com/paper/2509.13359/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 47 references that are in the map.
- First posted on arXiv 15 Sep 2025.

## Builds on (most cited first, 11 of 11)

- [Mathematical Capabilities of ChatGPT](https://mlsanity.com/paper/2301.13867/) (2023)
- [MathOdyssey: Benchmarking Mathematical Problem-Solving Skills in Large Language Models Using Odyssey Math Data](https://mlsanity.com/paper/2406.18321/) (2024)
- [Investigating the Effectiveness of ChatGPT in Mathematical Reasoning and Problem Solving: Evidence from the Vietnamese National High School Graduation Examination](https://mlsanity.com/paper/2306.06331/) (2023)
- [Evaluating GPT-4 at Grading Handwritten Solutions in Math Exams](https://mlsanity.com/paper/2411.05231/) (2024)
- [LLMs can Perform Multi-Dimensional Analytic Writing Assessments: A Case Study of L2 Graduate-Level Academic English Writing](https://mlsanity.com/paper/2502.11368/) (2025)
- [Three Questions Concerning the Use of Large Language Models to Facilitate Mathematics Learning](https://mlsanity.com/paper/2310.13615/) (2023)
- [Evaluation of LLMs for mathematical problem solving](https://mlsanity.com/paper/2506.00309/) (2025)
- [Assessing the Quality of AI-Generated Exams: A Large-Scale Field Study](https://mlsanity.com/paper/2508.08314/) (2025)
- [Generative AI Takes a Statistics Exam: A Comparison of Performance between ChatGPT3.5, ChatGPT4, and ChatGPT4o-mini](https://mlsanity.com/paper/2501.09171/) (2025)
- [Performance Comparison of Large Language Models on Advanced Calculus Problems](https://mlsanity.com/paper/2503.03960/) (2025)
- [Generative AI for Multiple Choice STEM Assessments](https://mlsanity.com/paper/2506.02094/) (2025)

## Abstract (arXiv)

Generative artificial intelligence (GenAI) tools such as OpenAI's ChatGPT are transforming the educational landscape, prompting reconsideration of traditional assessment practices. In parallel, universities are exploring alternatives to in-person, closed-book examinations, raising concerns about academic integrity and pedagogical alignment in uninvigilated settings. This study systematically investigates the performance of GenAI on typical mathematics questions from across a first-year mathematics curriculum. Adopting an empirical approach and utilising current examination questions as a proxy for course content, we generate, transcribe, and blind-mark GenAI submissions to eight undergraduate mathematics assessments, spanning the entirety of the first-year curriculum. By combining independent GenAI responses to individual questions, we enable a meaningful evaluation of GenAI performance, both at the level of modules and across the first-year curriculum. We find that GenAI attainment is at the level of a first-class degree, though current performance can vary between modules. Further, we find that GenAI performance is remarkably consistent when viewed across the entire curriculum, significantly more so than that of students in invigilated examinations. Our findings evidence the pressing need for redesigning assessments in mathematics in the era of generative artificial intelligence.