mlsanityData to 7 Oct 2026

Interactions with Prompt Problems: A New Way to Teach Programming with Large Language Models

arXiv 2401.10759 ↗First posted 19 Jan 2024cs.HC cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have upended decades of pedagogy in computing education. Students previously learned to code through textitwriting many small problems with less emphasis on code reading and comprehension. Recent research has shown that free code generation tools powered by LLMs can solve introductory programming problems presented in natural language with ease. In this paper, we propose a new way to teach programming with Prompt Problems. Students receive a problem visually, indicating how input should be transformed to output, and must translate that to a prompt for an LLM to decipher. The problem is considered correct when the code that is generated by the student prompt can pass all test cases. In this paper we present the design of this tool, discuss student interactions with it as they learn, and provide insights into this new class of programming problems as well as the design tools that integrate LLMs.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 19 of its 69 references that are in the map. First posted on arXiv 19 Jan 2024.

Builds on

19 of 69 references.

References · 19

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  5. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
  6. 2022Grounded Copilot: How Programmers Interact with Code-Generating Modelscited by 216
  7. 2022Programming Is Hard -- Or at Least It Used to Be: Educational Opportunities And Challenges of AI Code Generationcited by 97
  8. 2023Studying the effect of AI Code Generators on Supporting Novice Learners in Introductory Programmingcited by 87
  9. 2022Experiences from Using Code Explanations Generated by Large Language Models in a Web Software Development E-Bookcited by 84
  10. 2022Conversing with Copilot: Exploring Prompt Engineering for Solving CS1 Problems Using Natural Languagecited by 81
  11. 2022Using Large Language Models to Enhance Programming Error Messagescited by 67
  12. 2023Computing Education in the Era of Generative AIcited by 50
7 more
  1. 2023Exploring the Responses of Large Language Models to Beginner Programmers' Help Requestscited by 44
  2. 2023Thrilled by Your Progress! Large Language Models (GPT-4) No Longer Struggle to Pass Assessments in Higher Education Programming Coursescited by 38
  3. 2023Large Language Models (GPT) for automating feedback on programming assignmentscited by 32
  4. 2023Large Language Models (GPT) Struggle to Answer Multiple-Choice Questions about Codecited by 31
  5. 2023Can GPT-4 Support Analysis of Textual Data in Tasks Requiring Highly Specialized Domain Expertise?cited by 30
  6. 2023Exploring the Potential of Large Language Models to Generate Formative Programming Feedbackcited by 28
  7. 2023Harnessing LLMs in Curricular Design: Using GPT-4 to Support Authoring of Learning Objectivescited by 16

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024Substance Beats Style: Why Beginning Students Fail to Code with LLMscited by 5