mlsanityData to 7 Oct 2026

"It's not like Jarvis, but it's pretty close!" -- Examining ChatGPT's Usage among Undergraduate Students in Computer Science

arXiv 2311.09651 ↗First posted 16 Nov 2023cs.HC cs.AIcited by 9

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) such as ChatGPT and Google Bard have garnered significant attention in the academic community. Previous research has evaluated these LLMs for various applications such as generating programming exercises and solutions. However, these evaluations have predominantly been conducted by instructors and researchers, not considering the actual usage of LLMs by students. This study adopts a student-first approach to comprehensively understand how undergraduate computer science students utilize ChatGPT, a popular LLM, released by OpenAI. We employ a combination of student surveys and interviews to obtain valuable insights into the benefits, challenges, and suggested improvements related to ChatGPT. Our findings suggest that a majority of students (over 57%) have a convincingly positive outlook towards adopting ChatGPT as an aid in coursework-related tasks. However, our research also highlights various challenges that must be resolved for long-term acceptance of ChatGPT amongst students. The findings from this investigation have broader implications and may be applicable to other LLMs and their role in computing education.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 15 of its 48 references that are in the map. First posted on arXiv 16 Nov 2023.

Builds on

15 of 48 references.

References · 15

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2023Visual Instruction Tuningcited by 7,792
  3. 2022A Survey on In-context Learningcited by 717
  4. 2023A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPTcited by 612
  5. 2023Better Zero-Shot Reasoning with Role-Play Promptingcited by 220
  6. 2022Automatic Generation of Programming Exercises and Code Explanations using Large Language Modelscited by 124
  7. 2022Programming Is Hard -- Or at Least It Used to Be: Educational Opportunities And Challenges of AI Code Generationcited by 97
  8. 2022Experiences from Using Code Explanations Generated by Large Language Models in a Web Software Development E-Bookcited by 84
  9. 2022Conversing with Copilot: Exploring Prompt Engineering for Solving CS1 Problems Using Natural Languagecited by 81
  10. 2022Using Large Language Models to Enhance Programming Error Messagescited by 67
  11. 2023Comparing Code Explanations Created by Students and Large Language Modelscited by 63
  12. 2023On the Educational Impact of ChatGPT: Is Artificial Intelligence Ready to Obtain a University Degree?cited by 47
3 more
  1. 2023Can GPT-4 Support Analysis of Textual Data in Tasks Requiring Highly Specialized Domain Expertise?cited by 30
  2. 2023ChatGPT, Can You Generate Solutions for my Coding Exercises? An Evaluation on its Effectiveness in an undergraduate Java Programming Coursecited by 12
  3. 2020Effects of Human vs. Automatic Feedback on Students' Understanding of AI Concepts and Programming Stylecited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 1; 2026: 3.

9 papers.