# Ruffle&Riley: Towards the Automated Induction of Conversational Tutoring Systems

arXiv: https://arxiv.org/abs/2310.01420 · HTML: https://mlsanity.com/paper/2310.01420/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 66 references that are in the map.
- First posted on arXiv 26 Sep 2023.

## Builds on (most cited first, 7 of 7)

- [A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT](https://mlsanity.com/paper/2302.09419/) (2023)
- [CodeHelp: Using Large Language Models with Guardrails for Scalable Support in Programming Classes](https://mlsanity.com/paper/2308.06921/) (2023)
- [Learning gain differences between ChatGPT and human tutor generated algebra hints](https://mlsanity.com/paper/2302.06871/) (2023)
- [Assessing the Quality of Multiple-Choice Questions Using GPT-4 and Rule-Based Methods](https://mlsanity.com/paper/2307.08161/) (2023)
- [Evaluating ChatGPT's Decimal Skills and Feedback Generation in a Digital Learning Game](https://mlsanity.com/paper/2306.16639/) (2023)
- [Using Large Language Models to Provide Explanatory Feedback to Human Tutors](https://mlsanity.com/paper/2306.15498/) (2023)
- [Guidelines to Develop Trustworthy Conversational Agents for Children](https://mlsanity.com/paper/2209.02403/) (2022)

## Built on by (most cited first, 12 of 12)

- [Learning to Route LLMs with Confidence Tokens](https://mlsanity.com/paper/2410.13284/) (2024)
- [Can Large Language Models Match Tutoring System Adaptivity? A Benchmarking Study](https://mlsanity.com/paper/2504.05570/) (2025)
- [Evaluating and Optimizing Educational Content with Large Language Model Judgments](https://mlsanity.com/paper/2403.02795/) (2024)
- [SPL: A Socratic Playground for Learning Powered by Large Language Model](https://mlsanity.com/paper/2406.13919/) (2024)
- [Multilingual Performance Biases of Large Language Models in Education](https://mlsanity.com/paper/2504.17720/) (2025)
- [Predicting Learning Performance with Large Language Models: A Study in Adult Literacy](https://mlsanity.com/paper/2403.14668/) (2024)
- [StuGPTViz: A Visual Analytics Approach to Understand Student-ChatGPT Interactions](https://mlsanity.com/paper/2407.12423/) (2024)
- [Student Data Paradox and Curious Case of Single Student-Tutor Model: Regressive Side Effects of Training LLMs for Personalized Learning](https://mlsanity.com/paper/2404.15156/) (2024)
- [When LLMs Learn to be Students: The SOEI Framework for Modeling and Evaluating Virtual Student Agents in Educational Interaction](https://mlsanity.com/paper/2410.15701/) (2024)
- [Synthesizing High-Quality Programming Tasks with LLM-based Expert and Student Agents](https://mlsanity.com/paper/2504.07655/) (2025)
- [The Imitation Game for Educational AI](https://mlsanity.com/paper/2502.15127/) (2025)
- [Modularizing Educational LLM-Agency for Fostering Responsible Learning Assistance](https://mlsanity.com/paper/2605.30187/) (2026)

## Abstract (arXiv)

Conversational tutoring systems (CTSs) offer learning experiences driven by natural language interaction. They are known to promote high levels of cognitive engagement and benefit learning outcomes, particularly in reasoning tasks. Nonetheless, the time and cost required to author CTS content is a major obstacle to widespread adoption. In this paper, we introduce a novel type of CTS that leverages the recent advances in large language models (LLMs) in two ways: First, the system induces a tutoring script automatically from a lesson text. Second, the system automates the script orchestration via two LLM-based agents (Ruffle&Riley) with the roles of a student and a professor in a learning-by-teaching format. The system allows a free-form conversation that follows the ITS-typical inner and outer loop structure. In an initial between-subject online user study (N = 100) comparing Ruffle&Riley to simpler QA chatbots and reading activity, we found no significant differences in post-test scores. Nonetheless, in the learning experience survey, Ruffle&Riley users expressed higher ratings of understanding and remembering and further perceived the offered support as more helpful and the conversation as coherent. Our study provides insights for a new generation of scalable CTS technologies.