Assessing instructor-AI cooperation for grading essay-type questions in an introductory sociology course
Not on the map. Why?
Abstract (arXiv)
This study explores the use of artificial intelligence (AI) as a complementary tool for grading essay-type questions in higher education, focusing on its consistency with human grading and potential to reduce biases. Using 70 handwritten exams from an introductory sociology course, we evaluated generative pre-trained transformers (GPT) models' performance in transcribing and scoring students' responses. GPT models were tested under various settings for both transcription and grading tasks. Results show high similarity between human and GPT transcriptions, with GPT-4o-mini outperforming GPT-4o in accuracy. For grading, GPT demonstrated strong correlations with the human grader scores, especially when template answers were provided. However, discrepancies remained, highlighting GPT's role as a "second grader" to flag inconsistencies for assessment reviewing rather than fully replace human evaluation. This study contributes to the growing literature on AI in education, demonstrating its potential to enhance fairness and efficiency in grading essay-type questions.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 13 of its 80 references that are in the map. First posted on arXiv 11 Jan 2025.
Builds on
13 of 80 references.
References · 13
- 2022Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scalecited by 262
- 2023GPT detectors are biased against non-native English writerscited by 178
- 2023An Empirical Study of the Non-determinism of ChatGPT in Code Generationcited by 137
- 2023Social Biases through the Text-to-Image Generation Lenscited by 127
- 2023Generative Language Models Exhibit Social Identity Biasescited by 111
- 2024Measuring Political Bias in Large Language Models: What Is Said and How It Is Saidcited by 89
- 2023Bias Against 93 Stigmatized Groups in Masked Language Models and Downstream Sentiment Classification Taskscited by 51
- 2024Gender, Race, and Intersectional Bias in Resume Screening via Language Model Retrievalcited by 51
- 2023Bias of AI-Generated Content: An Examination of News Produced by Large Language Modelscited by 38
- 2023A Comprehensive AI Policy Education Framework for University Teaching and Learningcited by 34
- 2024Large Language Models Portray Socially Subordinate Groups as More Homogeneous, Consistent with a Bias Observed in Humanscited by 31
- 2024Bias in Generative AIcited by 19
Built on by
Citing papers per year: 2026: 1.
1 paper.