# Large Language Models in Computer Science Education: A Systematic Literature Review

arXiv: https://arxiv.org/abs/2410.16349 · HTML: https://mlsanity.com/paper/2410.16349/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 63 of its 138 references that are in the map.
- First posted on arXiv 21 Oct 2024.

## Builds on (most cited first, 30 of 63)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020)
- [A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly](https://mlsanity.com/paper/2312.02003/) (2023)
- [Automatic Generation of Programming Exercises and Code Explanations using Large Language Models](https://mlsanity.com/paper/2206.11861/) (2022)
- [Programming Is Hard -- Or at Least It Used to Be: Educational Opportunities And Challenges of AI Code Generation](https://mlsanity.com/paper/2212.01020/) (2022)
- [Studying the effect of AI Code Generators on Supporting Novice Learners in Introductory Programming](https://mlsanity.com/paper/2302.07427/) (2023)
- [The Robots are Here: Navigating the Generative AI Revolution in Computing Education](https://mlsanity.com/paper/2310.00658/) (2023)
- [Experiences from Using Code Explanations Generated by Large Language Models in a Web Software Development E-Book](https://mlsanity.com/paper/2211.02265/) (2022)
- [CodeAid: Evaluating a Classroom Deployment of an LLM-based Programming Assistant that Balances Student and Educator Needs](https://mlsanity.com/paper/2401.11314/) (2024)
- [Conversing with Copilot: Exploring Prompt Engineering for Solving CS1 Problems Using Natural Language](https://mlsanity.com/paper/2210.15157/) (2022)
- [CodeHelp: Using Large Language Models with Guardrails for Scalable Support in Programming Classes](https://mlsanity.com/paper/2308.06921/) (2023)
- [Comparing Code Explanations Created by Students and Large Language Models](https://mlsanity.com/paper/2304.03938/) (2023)
- [How Novices Use LLM-Based Code Generators to Solve CS1 Coding Tasks in a Self-Paced Learning Environment](https://mlsanity.com/paper/2309.14049/) (2023)
- [ChatGPT and Software Testing Education: Promises & Perils](https://mlsanity.com/paper/2302.03287/) (2023)
- [Thrilled by Your Progress! Large Language Models (GPT-4) No Longer Struggle to Pass Assessments in Higher Education Programming Courses](https://mlsanity.com/paper/2306.10073/) (2023)
- [Teach AI How to Code: Using Large Language Models as Teachable Agents for Programming Education](https://mlsanity.com/paper/2309.14534/) (2023)
- [A Comparative Study of AI-Generated (GPT-4) and Human-crafted MCQs in Programming Education](https://mlsanity.com/paper/2312.03173/) (2023)
- [StudentEval: A Benchmark of Student-Written Prompts for Large Language Models of Code](https://mlsanity.com/paper/2306.04556/) (2023)
- [Large Language Models (GPT) for automating feedback on programming assignments](https://mlsanity.com/paper/2307.00150/) (2023)
- [Prompt Problems: A New Programming Exercise for the Generative AI Era](https://mlsanity.com/paper/2311.05943/) (2023)
- [How Beginning Programmers and Code LLMs (Mis)read Each Other](https://mlsanity.com/paper/2401.15232/) (2024)
- [How to Teach Programming in the AI Era? Using LLMs as a Teachable Agent for Debugging](https://mlsanity.com/paper/2310.05292/) (2023)
- [An Empirical Study on Usage and Perceptions of LLMs in a Software Engineering Project](https://mlsanity.com/paper/2401.16186/) (2024)
- [Evaluating the Effectiveness of LLMs in Introductory Computer Science Education: A Semester-Long Field Study](https://mlsanity.com/paper/2404.13414/) (2024)
- [The Effects of Generative AI on Computing Students' Help-Seeking Preferences](https://mlsanity.com/paper/2401.02262/) (2024)
- [Next-Step Hint Generation for Introductory Programming Using Large Language Models](https://mlsanity.com/paper/2312.10055/) (2023)
- [Generative AI in Computing Education: Perspectives of Students and Instructors](https://mlsanity.com/paper/2308.04309/) (2023)

## Built on by (most cited first, 13 of 13)

- [TigerLLM - A Family of Bangla Large Language Models](https://mlsanity.com/paper/2503.10995/) (2025)
- [Beyond End-to-End Video Models: An LLM-Based Multi-Agent System for Educational Video Generation](https://mlsanity.com/paper/2602.11790/) (2026)
- [Code LLMs: A Taxonomy-based Survey](https://mlsanity.com/paper/2412.08291/) (2024)
- [Can LLMs Assist Computer Education? an Empirical Case Study of DeepSeek](https://mlsanity.com/paper/2504.00421/) (2025)
- [Learning by Teaching: Engaging Students as Instructors of Large Language Models in Computer Science Education](https://mlsanity.com/paper/2508.05979/) (2025)
- [Exploring Student Choice and the Use of Multimodal Generative AI in Programming Learning](https://mlsanity.com/paper/2510.05417/) (2025)
- [An Information-Theoretic Framework for Robust Large Language Model Editing](https://mlsanity.com/paper/2512.16227/) (2025)
- [Assessing the Software Security Comprehension of Large Language Models](https://mlsanity.com/paper/2512.21238/) (2025)
- [Kwame 2.0: Human-in-the-Loop Generative AI Teaching Assistant for Large Scale Online Coding Education in Africa](https://mlsanity.com/paper/2603.29159/) (2026)
- [Patterns of Learner-AI Interaction and Academic Performance in an Object-Oriented Programming Course](https://mlsanity.com/paper/2607.24755/) (2026)
- [Revisiting the Performance of Generative Artificial Intelligence on Introductory Object-Oriented Programming Assessments: Insights from 2026](https://mlsanity.com/paper/2608.16318/) (2026)
- [AlgoRAG: Retrieval-Augmented Generation for Theoretical Computer Science Education -- A Comprehensive Evaluation Framework for Algorithm Analysis and Complexity Theory](https://mlsanity.com/paper/2609.14572/) (2026)
- [SocraticRepair: Planning-Guided Socratic Tutoring for Program Debugging and Misconception Repair](https://mlsanity.com/paper/2610.15204/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are becoming increasingly better at a wide range of Natural Language Processing tasks (NLP), such as text generation and understanding. Recently, these models have extended their capabilities to coding tasks, bridging the gap between natural languages (NL) and programming languages (PL). Foundational models such as the Generative Pre-trained Transformer (GPT) and LLaMA series have set strong baseline performances in various NL and PL tasks. Additionally, several models have been fine-tuned specifically for code generation, showing significant improvements in code-related applications. Both foundational and fine-tuned models are increasingly used in education, helping students write, debug, and understand code. We present a comprehensive systematic literature review to examine the impact of LLMs in computer science and computer engineering education. We analyze their effectiveness in enhancing the learning experience, supporting personalized education, and aiding educators in curriculum development. We address five research questions to uncover insights into how LLMs contribute to educational outcomes, identify challenges, and suggest directions for future research.