mlsanityData to 7 Oct 2026

Programming Is Hard -- Or at Least It Used to Be: Educational Opportunities And Challenges of AI Code Generation

arXiv 2212.01020 ↗First posted 2 Dec 2022cs.HC cs.AI cs.CYcited by 97

Not on the map. Why?

Abstract (arXiv)

The introductory programming sequence has been the focus of much research in computing education. The recent advent of several viable and freely-available AI-driven code generation tools present several immediate opportunities and challenges in this domain. In this position paper we argue that the community needs to act quickly in deciding what possible opportunities can and should be leveraged and how, while also working on how to overcome or otherwise mitigate the possible challenges. Assuming that the effectiveness and proliferation of these tools will continue to progress rapidly, without quick, deliberate, and concerted efforts, educators will lose advantage in helping shape what opportunities come to be, and what challenges will endure. With this paper we aim to seed this discussion within the computing education community.

Key facts. Cited by 97 papers in the map (data through 7 Oct 2026). Builds on 11 of its 52 references that are in the map. First posted on arXiv 2 Dec 2022.

Builds on

11 of 52 references.

References · 11

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  4. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  5. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  6. 2021Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributionscited by 417
  7. 2021Understanding the Capabilities, Limitations, and Societal Impact of Large Language Modelscited by 159
  8. 2022GitHub Copilot AI pair programmer: Asset or Liability?cited by 150
  9. 2022Automatic Generation of Programming Exercises and Code Explanations using Large Language Modelscited by 124
  10. 2022Using Large Language Models to Enhance Programming Error Messagescited by 67
  11. 2022A Hazard Analysis Framework for Code Synthesis Large Language Modelscited by 22

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 29; 2024: 32; 2025: 24; 2026: 12.

97 papers.

Sort:

By area:

  1. 2023Self-Consuming Generative Models Go MADcited by 239
  2. 2023A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directionscited by 101
  3. 2023The Robots are Here: Navigating the Generative AI Revolution in Computing Educationcited by 85
  4. 2024CodeAid: Evaluating a Classroom Deployment of an LLM-based Programming Assistant that Balances Student and Educator Needscited by 81
  5. 2023Is ChatGPT the Ultimate Programming Assistant -- How far is it?cited by 72
  6. 2023Comparing Code Explanations Created by Students and Large Language Modelscited by 63
  7. 2023"It's Weird That it Knows What I Want": Usability and Interactions with Copilot for Novice Programmerscited by 62
  8. 2023Computing Education in the Era of Generative AIcited by 50
  9. 2024The Widening Gap: The Benefits and Harms of Generative AI for Novice Programmerscited by 46
  10. 2023Exploring the Responses of Large Language Models to Beginner Programmers' Help Requestscited by 44
  11. 2023Thrilled by Your Progress! Large Language Models (GPT-4) No Longer Struggle to Pass Assessments in Higher Education Programming Coursescited by 38
  12. 2024DetectRL: Benchmarking LLM-Generated Text Detection in Real-World Scenarioscited by 37
38 more
  1. 2023Large Language Models (GPT) Struggle to Answer Multiple-Choice Questions about Codecited by 31
  2. 2023Can GPT-4 Support Analysis of Textual Data in Tasks Requiring Highly Specialized Domain Expertise?cited by 30
  3. 2023How to Teach Programming in the AI Era? Using LLMs as a Teachable Agent for Debuggingcited by 29
  4. 2023Exploring the Potential of Large Language Models to Generate Formative Programming Feedbackcited by 28
  5. 2023Is AI the better programming partner? Human-Human Pair Programming vs. Human-AI pAIr Programmingcited by 20
  6. 2023Next-Step Hint Generation for Introductory Programming Using Large Language Modelscited by 20
  7. 2026In Line with Context: Repository-Level Code Generation via Context Inliningcited by 18
  8. 2023Promptly: Using Prompt Problems to Teach Learners How to Effectively Utilize AI Code Generatorscited by 16
  9. 2024Exploring the Design Space of Cognitive Engagement Techniques with AI-Generated Code for Enhanced Learningcited by 16
  10. 2023ChatGPT in the Classroom: An Analysis of Its Strengths and Weaknesses for Solving Undergraduate Computer Science Questionscited by 13
  11. 2024Evaluating Contextually Personalized Programming Exercises Created with Generative AIcited by 13
  12. 2024Beyond the Hype: A Comprehensive Review of Current Trends in Generative AI Research, Teaching Practices, and Toolscited by 13
  13. 2023Can We Trust AI-Generated Educational Content? Comparative Analysis of Human and AI-Generated Learning Resourcescited by 12
  14. 2024Large Language Models in Computer Science Education: A Systematic Literature Reviewcited by 12
  15. 2023Dcc --help: Generating Context-Aware Compiler Error Explanations with Large Language Modelscited by 11
  16. 2025Conversational AI as a Coding Assistant: Understanding Programmers' Interactions with and Expectations from Large Language Models for Codingcited by 11
  17. 2023Large Language Models in Introductory Programming Education: ChatGPT's Performance and Implications for Assessmentscited by 10
  18. 2023More Robots are Coming: Large Multimodal Models (ChatGPT) can Solve Visually Diverse Images of Parsons Problemscited by 10
  19. 2024Generative AI in Education: A Study of Educators' Awareness, Sentiments, and Influencing Factorscited by 10
  20. 2024Could ChatGPT get an Engineering Degree? Evaluating Higher Education Vulnerability to AI Assistantscited by 10
  21. 2025Security Degradation in Iterative AI Code Generation -- A Systematic Analysis of the Paradoxcited by 10
  22. 2023"It's not like Jarvis, but it's pretty close!" -- Examining ChatGPT's Usage among Undergraduate Students in Computer Sciencecited by 9
  23. 2024Iris: An AI-Driven Virtual Tutor For Computer Science Educationcited by 9
  24. 2024Insights from Social Shaping Theory: The Appropriation of Large Language Models in an Undergraduate Programming Coursecited by 9
  25. 2025How Do Programming Students Use Generative AI?cited by 9
  26. 2023Seeing Seeds Beyond Weeds: Green Teaming Generative AI for Beneficial Usescited by 8
  27. 2023AI-enhanced Auto-correction of Programming Exercises: How Effective is GPT-3.5?cited by 8
  28. 2025UPME: An Unsupervised Peer Review Framework for Multimodal Large Language Model Evaluationcited by 7
  29. 2024Validating LLM-Generated Programs with Metamorphic Prompt Testingcited by 6
  30. 2025Student Perspectives on the Benefits and Risks of AI in Educationcited by 6
  31. 2024MAiDE-up: Multilingual Deception Detection of GPT-generated Hotel Reviewscited by 5
  32. 2024Evaluating AI-generated code for C++, Fortran, Go, Java, Julia, Matlab, Python, R, and Rustcited by 5
  33. 2024Students' Perceptions and Use of Generative AI Tools for Programming Across Different Computing Coursescited by 5
  34. 2024AIGCodeSet: A New Annotated Dataset for AI Generated Code Detectioncited by 5
  35. 2025TableTalk: Scaffolding Spreadsheet Development with a Language Agentcited by 5
  36. 2023Kattis vs. ChatGPT: Assessment and Evaluation of Programming Tasks in the Age of Artificial Intelligencecited by 4
  37. 2024Large Language Models Meet User Interfaces: The Case of Provisioning Feedbackcited by 4
  38. 2024From An LLM Swarm To A PDDL-Empowered HIVE: Planning Self-Executed Instructions In A Multi-Modal Junglecited by 4

50 most cited of 97 All 97 (JSON)