# Evaluating ChatGPT's Decimal Skills and Feedback Generation in a Digital Learning Game

arXiv: https://arxiv.org/abs/2306.16639 · HTML: https://mlsanity.com/paper/2306.16639/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 34 references that are in the map.
- First posted on arXiv 29 Jun 2023.

## Builds on (most cited first, 6 of 6)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [Is ChatGPT a General-Purpose Natural Language Processing Task Solver?](https://mlsanity.com/paper/2302.06476/) (2023)
- [Mathematical Capabilities of ChatGPT](https://mlsanity.com/paper/2301.13867/) (2023)
- [A Comprehensive Capability Analysis of GPT-3 and GPT-3.5 Series Models](https://mlsanity.com/paper/2303.10420/) (2023)
- [Learning gain differences between ChatGPT and human tutor generated algebra hints](https://mlsanity.com/paper/2302.06871/) (2023)

## Built on by (most cited first, 19 of 19)

- [Ruffle&Riley: Insights from Designing and Evaluating a Large Language Model-Based Conversational Tutoring System](https://mlsanity.com/paper/2404.17460/) (2024)
- [Improving the Validity of Automatically Generated Feedback via Reinforcement Learning](https://mlsanity.com/paper/2403.01304/) (2024)
- [Ruffle&Riley: Towards the Automated Induction of Conversational Tutoring Systems](https://mlsanity.com/paper/2310.01420/) (2023)
- [GPT for Games: A Scoping Review (2020-2023)](https://mlsanity.com/paper/2404.17794/) (2024)
- [GPT for Games: An Updated Scoping Review (2020-2024)](https://mlsanity.com/paper/2411.00308/) (2024)
- [PromptHive: Bringing Subject Matter Experts Back to the Forefront with Collaborative Prompt Engineering for Educational Content Creation](https://mlsanity.com/paper/2410.16547/) (2024)
- [Automatic Generation of Question Hints for Mathematics Problems using Large Language Models in Educational Technology](https://mlsanity.com/paper/2411.03495/) (2024)
- [A Systematic Review on Prompt Engineering in Large Language Models for K-12 STEM Education](https://mlsanity.com/paper/2410.11123/) (2024)
- [Towards Adaptive Feedback with AI: Comparing the Feedback Quality of LLMs and Teachers on Experimentation Protocols](https://mlsanity.com/paper/2502.12842/) (2025)
- [CoT Information: Improved Sample Complexity under Chain-of-Thought Supervision](https://mlsanity.com/paper/2505.15927/) (2025)
- [Large Language Models Meet User Interfaces: The Case of Provisioning Feedback](https://mlsanity.com/paper/2404.11072/) (2024)
- [Can Large Language Models Replicate ITS Feedback on Open-Ended Math Questions?](https://mlsanity.com/paper/2405.06414/) (2024)
- [Automated Feedback in Math Education: A Comparative Analysis of LLMs for Open-Ended Responses](https://mlsanity.com/paper/2411.08910/) (2024)
- [Use Me Wisely: AI-Driven Assessment for LLM Prompting Skills Development](https://mlsanity.com/paper/2503.02532/) (2025)
- [Automatic Large Language Models Creation of Interactive Learning Lessons](https://mlsanity.com/paper/2506.17356/) (2025)
- [Comparing RAG and GraphRAG for Page-Level Retrieval Question Answering on a Math Textbook](https://mlsanity.com/paper/2509.16780/) (2025)
- [Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module](https://mlsanity.com/paper/2508.13962/) (2025)
- [REFINE: Real-world Exploration of Interactive Feedback and Student Behaviour](https://mlsanity.com/paper/2603.29142/) (2026)
- [Learning to Optimize Feedback for One Million Students: Insights from Multi-Armed and Contextual Bandits in Large-Scale Online Tutoring](https://mlsanity.com/paper/2508.00270/) (2025)

## Abstract (arXiv)

While open-ended self-explanations have been shown to promote robust learning in multiple studies, they pose significant challenges to automated grading and feedback in technology-enhanced learning, due to the unconstrained nature of the students' input. Our work investigates whether recent advances in Large Language Models, and in particular ChatGPT, can address this issue. Using decimal exercises and student data from a prior study of the learning game Decimal Point, with more than 5,000 open-ended self-explanation responses, we investigate ChatGPT's capability in (1) solving the in-game exercises, (2) determining the correctness of students' answers, and (3) providing meaningful feedback to incorrect answers. Our results showed that ChatGPT can respond well to conceptual questions, but struggled with decimal place values and number line problems. In addition, it was able to accurately assess the correctness of 75% of the students' answers and generated generally high-quality feedback, similar to human instructors. We conclude with a discussion of ChatGPT's strengths and weaknesses and suggest several venues for extending its use cases in digital teaching and learning.