mlsanityData to 14 Oct 2026

Can ChatGPT Pass An Introductory Level Functional Language Programming Course?

arXiv 2305.02230 ↗First posted 29 Apr 2023cs.CY cs.AIcited by 9

Not on the map. Why?

Abstract (arXiv)

The recent introduction of ChatGPT has drawn significant attention from both industry and academia due to its impressive capabilities in solving a diverse range of tasks, including language translation, text summarization, and computer programming. Its capability for writing, modifying, and even correcting code together with its ease of use and access is already dramatically impacting computer science education. This paper aims to explore how well ChatGPT can perform in an introductory-level functional language programming course. In our systematic evaluation, we treated ChatGPT as one of our students and demonstrated that it can achieve a grade B- and its rank in the class is 155 out of 314 students overall. Our comprehensive evaluation provides valuable insights into ChatGPT's impact from both student and instructor perspectives. Additionally, we identify several potential benefits that ChatGPT can offer to both groups. Overall, we believe that this study significantly clarifies and advances our understanding of ChatGPT's capabilities and potential impact on computer science education.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 24 of its 52 references that are in the map. First posted on arXiv 29 Apr 2023.

Builds on

24 of 52 references.

References · 24

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  7. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  8. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,310
  9. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
  10. 2023A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPTcited by 619
  11. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 606
  12. 2022FOLIO: Natural Language Reasoning with First-Order Logiccited by 235
12 more
  1. 2015A Critical Review of Recurrent Neural Networks for Sequence Learningcited by 224
  2. 2022Grounded Copilot: How Programmers Interact with Code-Generating Modelscited by 218
  3. 2023Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Modelscited by 154
  4. 2023An Analysis of the Automatic Bug Fixing Performance of ChatGPTcited by 110
  5. 2023Impact of Code Language Models on Automated Program Repaircited by 85
  6. 2023Is ChatGPT the Ultimate Programming Assistant -- How far is it?cited by 76
  7. 2022Interactive and Visual Prompt Engineering for Ad-hoc Task Adaptation with Large Language Modelscited by 70
  8. 2022Is GitHub's Copilot as Bad as Humans at Introducing Vulnerabilities in Code?cited by 67
  9. 2021CodeTrans: Towards Cracking the Language of Silicon's Code Through Self-Supervised Deep Learning and High Performance Computingcited by 28
  10. 2022How Readable is Model-generated Code? Examining Readability and Visual Inspection of GitHub Copilotcited by 14
  11. 2020Mossad: Defeating Software Plagiarism Detectioncited by 5
  12. 2023Identifying Different Student Clusters in Functional Programming Assignments: From Quick Learners to Struggling Studentscited by 1

Built on by

Cited by year

Citing papers per year: 2023: 4; 2024: 4; 2025: 1.

9 papers.

  1. 2024How Novice Programmers Use and Experience ChatGPT when Solving Programming Exercises in an Introductory Coursecited by 11
  2. 2023AI-enhanced Auto-correction of Programming Exercises: How Effective is GPT-3.5?cited by 8
  3. 2024Analyzing Chat Protocols of Novice Programmers Solving Introductory Programming Tasks with ChatGPTcited by 8
  4. 2023Kattis vs. ChatGPT: Assessment and Evaluation of Programming Tasks in the Age of Artificial Intelligencecited by 4
  5. 2023Student Mastery or AI Deception? Analyzing ChatGPT's Assessment Proficiency and Evaluating Detection Strategiescited by 1
  6. 2024Malinowski in the Age of AI: Can large language models create a text game based on an anthropological classic?cited by 1
  7. 2025Students' Feedback Requests and Interactions with the SCRIPT Chatbot: Do They Get What They Ask For?cited by 1
  8. 2023ChatGPT & Mechanical Engineering: Examining performance on the FE Mechanical Engineering and Undergraduate Examscited by 0
  9. 2024Can OpenSource beat ChatGPT? -- A Comparative Study of Large Language Models for Text-to-Code Generationcited by 0