# AI Meets Mathematics Education: A Case Study on Supporting an Instructor in a Large Mathematics Class with Context-Aware AI

arXiv: https://arxiv.org/abs/2603.26679 · HTML: https://mlsanity.com/paper/2603.26679/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 65 references that are in the map.
- First posted on arXiv 9 Mar 2026.

## Builds on (most cited first, 22 of 22)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [Humans or LLMs as the Judge? A Study on Judgement Biases](https://mlsanity.com/paper/2402.10669/) (2024)
- [OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset](https://mlsanity.com/paper/2402.10176/) (2024)
- [MathTutorBench: A Benchmark for Measuring Open-ended Pedagogical Capabilities of LLM Tutors](https://mlsanity.com/paper/2502.18940/) (2025)
- [AI-Augmented Brainwriting: Investigating the use of LLMs in group ideation](https://mlsanity.com/paper/2402.14978/) (2024)
- [Application-Driven Innovation in Machine Learning](https://mlsanity.com/paper/2403.17381/) (2024)
- [U-MATH: A University-Level Benchmark for Evaluating Mathematical Skills in LLMs](https://mlsanity.com/paper/2412.03205/) (2024)
- [A LLM-Powered Automatic Grading Framework with Human-Level Guidelines Optimization](https://mlsanity.com/paper/2410.02165/) (2024)
- [FairAIED: Navigating Fairness, Bias, and Ethics in Educational AI Applications](https://mlsanity.com/paper/2407.18745/) (2024)
- [Stop treating `AGI' as the north-star goal of AI research](https://mlsanity.com/paper/2502.03689/) (2025)
- [TutorUp: What If Your Students Were Simulated? Training Tutors to Address Engagement Challenges in Online Learning](https://mlsanity.com/paper/2502.16178/) (2025)
- [AI Mismatches: Identifying Potential Algorithmic Harms Before AI Development](https://mlsanity.com/paper/2502.18682/) (2025)
- [BIPED: Pedagogically Informed Tutoring System for ESL Education](https://mlsanity.com/paper/2406.03486/) (2024)
- [Assessing the efficacy of large language models in generating accurate teacher responses](https://mlsanity.com/paper/2307.04274/) (2023)
- [Developing a Tutoring Dialog Dataset to Optimize LLMs for Educational Use](https://mlsanity.com/paper/2410.19231/) (2024)
- [Co-designing Large Language Model Tools for Project-Based Learning with K12 Educators](https://mlsanity.com/paper/2502.09799/) (2025)
- [Effective and Scalable Math Support: Evidence on the Impact of an AI- Tutor on Math Achievement in Ghana](https://mlsanity.com/paper/2402.09809/) (2024)
- [From Passive Watching to Active Learning: Empowering Proactive Participation in Digital Classrooms with AI Video Assistant](https://mlsanity.com/paper/2409.15843/) (2024)
- ["Would You Want an AI Tutor?" Understanding Stakeholder Perceptions of LLM-based Systems in the Classroom](https://mlsanity.com/paper/2503.02885/) (2025)
- [Do AI tutors empower or enslave learners? Toward a critical use of AI in education](https://mlsanity.com/paper/2507.06878/) (2025)
- [Potential Societal Biases of ChatGPT in Higher Education: A Scoping Review](https://mlsanity.com/paper/2311.14381/) (2023)

## Abstract (arXiv)

Large-enrollment university courses face persistent challenges in providing timely and scalable instructional support. While generative AI holds promise, its effective use depends on reliability and pedagogical alignment. We present a human-centered case study of AI-assisted support in a Calculus I course, implemented in close collaboration with the course instructor. We developed a system to answer students' questions on a discussion forum, fine-tuning a lightweight language model on 2,588 historical student-instructor interactions. The model achieved 75.3% accuracy on a benchmark of 150 representative questions annotated by five instructors, and in 36% of cases, its responses were rated equal to or better than instructor answers. Post-deployment student survey (N = 105) indicated that students valued the alignment of the responses with the course materials and their immediate availability, while still relying on the instructor verification for trust. We highlight the importance of hybrid human-AI workflows for safe and effective course support.