ChatGPT as a Solver and Grader of Programming Exams written in Spanish
Not on the map. Why?
Abstract (arXiv)
Evaluating the capabilities of Large Language Models (LLMs) to assist teachers and students in educational tasks is receiving increasing attention. In this paper, we assess ChatGPT's capacities to solve and grade real programming exams, from an accredited BSc degree in Computer Science, written in Spanish. Our findings suggest that this AI model is only effective for solving simple coding tasks. Its proficiency in tackling complex problems or evaluating solutions authored by others are far from effective. As part of this research, we also release a new corpus of programming tasks and the corresponding prompts for solving the problems or grading the solutions. This resource can be further exploited by other research teams.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 9 of its 14 references that are in the map. First posted on arXiv 23 Sep 2024.
Builds on
9 of 14 references.
References · 9
- 2017Attention Is All You Needcited by 59,932
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
- 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
- 2021Measuring Coding Challenge Competence With APPScited by 879
- 2022A Systematic Evaluation of Large Language Models of Codecited by 360
- 2023Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned BERTcited by 155
- 2023GPTEval: A Survey on Assessments of ChatGPT and GPT-4cited by 73
Built on by
Not cited yet.