# AI2T: Building Trustable AI Tutors by Interactively Teaching a Self-Aware Learning Agent

arXiv: https://arxiv.org/abs/2411.17924 · HTML: https://mlsanity.com/paper/2411.17924/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Educational agents and math problem solving, one of 16 directions in LLM agents and tool use.
- Builds on 7 of its 59 references that are in the map.
- First posted on arXiv 26 Nov 2024.

## Placed in

- LLM agents and tool use / [Educational agents and math problem solving](https://mlsanity.com/agents/educational-agents-and-math-problem-solving/)

## Builds on (most cited first, 7 of 7)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [VAL: Interactive Task Learning with GPT Dialog Parsing](https://mlsanity.com/paper/2310.01627/) (2023)
- [WebRobot: Web Robotic Process Automation using Interactive Programming-by-Demonstration](https://mlsanity.com/paper/2203.09993/) (2022)
- [DiLogics: Creating Web Automation Programs With Diverse Logics](https://mlsanity.com/paper/2308.05828/) (2023)
- [Towards Educator-Driven Tutor Authoring: Generative AI Approaches for Creating Intelligent Tutor Interfaces](https://mlsanity.com/paper/2405.14713/) (2024)
- [HTN-Based Tutors: A New Intelligent Tutoring Framework Based on Hierarchical Task Networks](https://mlsanity.com/paper/2405.14716/) (2024)
- [STAND: Self-Aware Precondition Induction for Interactive Task Learning](https://mlsanity.com/paper/2409.07653/) (2024)

## Built on by (most cited first, 5 of 5)

- [TutorGym: A Testbed for Evaluating AI Agents as Tutors and Students](https://mlsanity.com/paper/2505.01563/) (2025)
- [Model Human Learners: Computational Models to Guide Instructional Design](https://mlsanity.com/paper/2502.02456/) (2025)
- [Toward a Benchmark for Controllable Simulation of Imperfect Students with Large Language Models](https://mlsanity.com/paper/2605.25601/) (2026)
- [SMTB: Fast Structure-Mapping with Tight Bounds](https://mlsanity.com/paper/2609.25508/) (2026)
- [Decomposed Inductive Procedure Learning: Learning Academic Tasks with Human-Like Data Efficiency](https://mlsanity.com/paper/2505.10422/) (2025)

## Abstract (arXiv)

AI2T is an interactively teachable AI for authoring intelligent tutoring systems (ITSs). Authors tutor AI2T by providing a few step-by-step solutions and then grading AI2T's own problem-solving attempts. From just 20-30 minutes of interactive training, AI2T can induce robust rules for step-by-step solution tracking (i.e., model-tracing). As AI2T learns it can accurately estimate its certainty of performing correctly on unseen problem steps using STAND: a self-aware precondition learning algorithm that outperforms state-of-the-art methods like XGBoost. Our user study shows that authors can use STAND's certainty heuristic to estimate when AI2T has been trained on enough diverse problems to induce correct and complete model-tracing programs. AI2T-induced programs are more reliable than hallucination-prone LLMs and prior authoring-by-tutoring approaches. With its self-aware induction of hierarchical rules, AI2T offers a path toward trustable data-efficient authoring-by-tutoring for complex ITSs that normally require as many as 200-300 hours of programming per hour of instruction.