# StuGPTViz: A Visual Analytics Approach to Understand Student-ChatGPT Interactions

arXiv: https://arxiv.org/abs/2407.12423 · HTML: https://mlsanity.com/paper/2407.12423/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 78 references that are in the map.
- First posted on arXiv 17 Jul 2024.

## Builds on (most cited first, 15 of 15)

- [A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT](https://mlsanity.com/paper/2302.11382/) (2023)
- [LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset](https://mlsanity.com/paper/2309.11998/) (2023)
- [Ragas: Automated Evaluation of Retrieval Augmented Generation](https://mlsanity.com/paper/2309.15217/) (2023)
- [Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond](https://mlsanity.com/paper/2304.13712/) (2023)
- [ChatGPT: Jack of all trades, master of none](https://mlsanity.com/paper/2302.10724/) (2023)
- [TheoremQA: A Theorem-driven Question Answering dataset](https://mlsanity.com/paper/2305.12524/) (2023)
- [ChatGPT Prompt Patterns for Improving Code Quality, Refactoring, Requirements Elicitation, and Software Design](https://mlsanity.com/paper/2303.07839/) (2023)
- [Large Language Models in Education: Vision and Opportunities](https://mlsanity.com/paper/2311.13160/) (2023)
- [GPT-3-driven pedagogical agents for training children's curious question-asking skills](https://mlsanity.com/paper/2211.14228/) (2022)
- [Impact of Guidance and Interaction Strategies for LLM Use on Learner Performance and Perception](https://mlsanity.com/paper/2310.13712/) (2023)
- [Ruffle&Riley: Towards the Automated Induction of Conversational Tutoring Systems](https://mlsanity.com/paper/2310.01420/) (2023)
- [Challenges and Opportunities in Data Visualization Education: A Call to Action](https://mlsanity.com/paper/2308.07703/) (2023)
- [Adaptive Assessment of Visualization Literacy](https://mlsanity.com/paper/2308.14147/) (2023)
- [ConVIScope: Visual Analytics for Exploring Patient Conversations](https://mlsanity.com/paper/2108.13514/) (2021)
- [Learning Vis Tools: Teaching Data Visualization Tutorials](https://mlsanity.com/paper/1907.08796/) (2019)

## Built on by (most cited first, 7 of 7)

- [WildVis: Open Source Visualizer for Million-Scale Chat Logs in the Wild](https://mlsanity.com/paper/2409.03753/) (2024)
- [OnGoal: Tracking and Visualizing Conversational Goals in Multi-Turn Dialogue with Large Language Models](https://mlsanity.com/paper/2508.21061/) (2025)
- [A Deep User Interface for Exploring LLaMa](https://mlsanity.com/paper/2502.20938/) (2025)
- [A Showdown of ChatGPT vs DeepSeek in Solving Programming Tasks](https://mlsanity.com/paper/2503.13549/) (2025)
- [Are Agents Ready to Teach? A Multi-Stage Benchmark for Real-World Teaching Workflows](https://mlsanity.com/paper/2605.14322/) (2026)
- [AI-Assisted Help-Seeking Trajectories in Programming Education from an SRL-Informed Perspective](https://mlsanity.com/paper/2606.22809/) (2026)
- [Walking through Discussions: A Mobile Visual Analytics System for In-Situ Group Discussion Analysis](https://mlsanity.com/paper/2608.08617/) (2026)

## Abstract (arXiv)

The integration of Large Language Models (LLMs), especially ChatGPT, into education is poised to revolutionize students' learning experiences by introducing innovative conversational learning methodologies. To empower students to fully leverage the capabilities of ChatGPT in educational scenarios, understanding students' interaction patterns with ChatGPT is crucial for instructors. However, this endeavor is challenging due to the absence of datasets focused on student-ChatGPT conversations and the complexities in identifying and analyzing the evolutional interaction patterns within conversations. To address these challenges, we collected conversational data from 48 students interacting with ChatGPT in a master's level data visualization course over one semester. We then developed a coding scheme, grounded in the literature on cognitive levels and thematic analysis, to categorize students' interaction patterns with ChatGPT. Furthermore, we present a visual analytics system, StuGPTViz, that tracks and compares temporal patterns in student prompts and the quality of ChatGPT's responses at multiple scales, revealing significant pedagogical insights for instructors. We validated the system's effectiveness through expert interviews with six data visualization instructors and three case studies. The results confirmed StuGPTViz's capacity to enhance educators' insights into the pedagogical value of ChatGPT. We also discussed the potential research opportunities of applying visual analytics in education and developing AI-driven personalized learning solutions.