# Navigating Pitfalls: Evaluating LLMs in Machine Learning Programming Education

arXiv: https://arxiv.org/abs/2505.18220 · HTML: https://mlsanity.com/paper/2505.18220/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 55 references that are in the map.
- First posted on arXiv 23 May 2025.

## Builds on (most cited first, 25 of 25)

- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [A Survey on Large Language Models for Code Generation](https://mlsanity.com/paper/2406.00515/) (2024)
- [A Comprehensive Overview of Large Language Models](https://mlsanity.com/paper/2307.06435/) (2023)
- [Practical Program Repair in the Era of Large Pre-trained Language Models](https://mlsanity.com/paper/2210.14179/) (2022)
- [An Analysis of the Automatic Bug Fixing Performance of ChatGPT](https://mlsanity.com/paper/2301.08653/) (2023)
- [Programming Is Hard -- Or at Least It Used to Be: Educational Opportunities And Challenges of AI Code Generation](https://mlsanity.com/paper/2212.01020/) (2022)
- [The Robots are Here: Navigating the Generative AI Revolution in Computing Education](https://mlsanity.com/paper/2310.00658/) (2023)
- [Experiences from Using Code Explanations Generated by Large Language Models in a Web Software Development E-Book](https://mlsanity.com/paper/2211.02265/) (2022)
- [Machine Learning in Python: Main developments and technology trends in data science, machine learning, and artificial intelligence](https://mlsanity.com/paper/2002.04803/) (2020)
- [Is GPT-4 a Good Data Analyst?](https://mlsanity.com/paper/2305.15038/) (2023)
- ["It's Weird That it Knows What I Want": Usability and Interactions with Copilot for Novice Programmers](https://mlsanity.com/paper/2304.02491/) (2023)
- [Bugs in Large Language Models Generated Code: An Empirical Study](https://mlsanity.com/paper/2403.08937/) (2024)
- [Forecast Evaluation for Data Scientists: Common Pitfalls and Best Practices](https://mlsanity.com/paper/2203.10716/) (2022)
- [Exploring the Responses of Large Language Models to Beginner Programmers' Help Requests](https://mlsanity.com/paper/2306.05715/) (2023)
- [How to avoid machine learning pitfalls: a guide for academic researchers](https://mlsanity.com/paper/2108.02497/) (2021)
- [Generating High-Precision Feedback for Programming Syntax Errors using Large Language Models](https://mlsanity.com/paper/2302.04662/) (2023)
- [LLMs for Science: Usage for Code Generation and Data Analysis](https://mlsanity.com/paper/2311.16733/) (2023)
- [Repairing Bugs in Python Assignments Using Large Language Models](https://mlsanity.com/paper/2209.14876/) (2022)
- [Evaluating the Effectiveness of LLMs in Introductory Computer Science Education: A Semester-Long Field Study](https://mlsanity.com/paper/2404.13414/) (2024)
- [A Meta-Summary of Challenges in Building Products with ML Components -- Collecting Experiences from 4758+ Practitioners](https://mlsanity.com/paper/2304.00078/) (2023)
- [Automated Clustering and Program Repair for Introductory Programming Assignments](https://mlsanity.com/paper/1603.03165/) (2016)
- [Desirable Characteristics for AI Teaching Assistants in Programming Education](https://mlsanity.com/paper/2405.14178/) (2024)
- [What Should Data Science Education Do with Large Language Models?](https://mlsanity.com/paper/2307.02792/) (2023)
- [LLM-itation is the Sincerest Form of Data: Generating Synthetic Buggy Code Submissions for Computing Education](https://mlsanity.com/paper/2411.10455/) (2024)

## Built on by (most cited first, 2 of 2)

- [Open Tabular Insight Extraction: Where Do We Stand, and Where Should We Go?](https://mlsanity.com/paper/2609.07629/) (2026)
- [The Silent Failures in Agentic Data Science](https://mlsanity.com/paper/2610.14115/) (2026)

## Abstract (arXiv)

The rapid advancement of Large Language Models (LLMs) has opened new avenues in education. This study examines the use of LLMs in supporting learning in machine learning education; in particular, it focuses on the ability of LLMs to identify common errors of practice (pitfalls) in machine learning code, and their ability to provide feedback that can guide learning. Using a portfolio of code samples, we consider four different LLMs: one closed model and three open models. Whilst the most basic pitfalls are readily identified by all models, many common pitfalls are not. They particularly struggle to identify pitfalls in the early stages of the ML pipeline, especially those which can lead to information leaks, a major source of failure within applied ML projects. They also exhibit limited success at identifying pitfalls around model selection, which is a concept that students often struggle with when first transitioning from theory to practice. This questions the use of current LLMs to support machine learning education, and also raises important questions about their use by novice practitioners. Nevertheless, when LLMs successfully identify pitfalls in code, they do provide feedback that includes advice on how to proceed, emphasising their potential role in guiding learners. We also compare the capability of closed and open LLM models, and find that the gap is relatively small given the large difference in model sizes. This presents an opportunity to deploy, and potentially customise, smaller more efficient LLM models within education, avoiding risks around cost and data sharing associated with commercial models.