VAL: Interactive Task Learning with GPT Dialog Parsing
Not on the map. Why?
Abstract (arXiv)
Machine learning often requires millions of examples to produce static, black-box models. In contrast, interactive task learning (ITL) emphasizes incremental knowledge acquisition from limited instruction provided by humans in modalities such as natural language. However, ITL systems often suffer from brittle, error-prone language parsing, which limits their usability. Large language models (LLMs) are resistant to brittleness but are not interpretable and cannot learn incrementally. We present VAL, an ITL system with a new philosophy for LLM/symbolic integration. By using LLMs only for specific tasks--such as predicate and argument selection--within an algorithmic framework, VAL reaps the benefits of LLMs to support interactive learning of hierarchical task knowledge from natural language. Acquired knowledge is human interpretable and generalizes to support execution of novel tasks without additional training. We studied users' interactions with VAL in a video game setting, finding that most users could successfully teach VAL using language they felt was natural.
Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 16 of its 51 references that are in the map. First posted on arXiv 2 Oct 2023.
Builds on
16 of 51 references.
References · 16
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
- 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
- 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
- 2022Scaling Laws for Reward Model Overoptimizationcited by 1,056
- 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
- 2015Compositional Semantic Parsing on Semi-Structured Tablescited by 509
- 2019On the Utility of Learning about Humans for Human-AI Coordinationcited by 345
- 2021AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Promptscited by 265
- 2022PromptChainer: Chaining Large Language Model Prompts through Visual Programmingcited by 134
- 2023ProAgent: Building Proactive Cooperative Agents with Large Language Modelscited by 97
4 more
- 2019Interactive Task and Concept Learning from Natural Language Instructions and GUI Demonstrationscited by 26
- 2023COACH: Cooperative Robot Teachingcited by 7
- 2021Unpacking Human Teachers' Intentions For Natural Interactive Task Learningcited by 6
- 2022Integrating Diverse Knowledge Sources for Online One-shot Learning of Novel Taskscited by 6
Built on by
Citing papers per year: 2024: 5; 2025: 9.
14 papers.
Sort:
By area:
- 2024Cocoa: Co-Planning and Co-Execution with AI Agentscited by 38
- 2024Large Language Models and Video Games: A Preliminary Scoping Reviewcited by 23
- 2025Understanding the LLM-ification of CHI: Unpacking the Impact of LLMs at CHI through a Systematic Literature Reviewcited by 16
- 2024GPT for Games: An Updated Scoping Review (2020-2024)cited by 10
- 2025SPHERE: An Evaluation Card for Human-AI Systemscited by 9
- 2025When Should Users Check? Modeling Confirmation Frequency in Multi-Step Agentic AI Taskscited by 6
- 2024AI2T: Building Trustable AI Tutors by Interactively Teaching a Self-Aware Learning Agentcited by 5
- 2025AgentSense: Virtual Sensor Data Generation Using LLM Agents in Simulated Home Environmentscited by 4
- 2025Interactive Program Synthesis for Modeling Collaborative Physical Activities from Narrated Demonstrationscited by 2
- 2024STAND: Self-Aware Precondition Induction for Interactive Task Learningcited by 1
- 2025Online Learning of HTN Methods for integrated LLM-HTN Planningcited by 1
- 2025Challenges in Grounding Language in the Real Worldcited by 0
- 2025Online Learning of HTN Methods for integrated LLM-HTN Planning17 Nov 2025
- 2025OrchVis: Hierarchical Multi-Agent Orchestration for Human Oversight28 Oct 2025
- 2025When Should Users Check? Modeling Confirmation Frequency in Multi-Step Agentic AI Tasks6 Oct 2025
- 2025Interactive Program Synthesis for Modeling Collaborative Physical Activities from Narrated Demonstrations29 Sep 2025
- 2025Using Natural Language for Human-Robot Collaboration in the Real World15 Aug 2025
- 2025Challenges in Grounding Language in the Real World20 Jun 2025
- 2025AgentSense: Virtual Sensor Data Generation Using LLM Agents in Simulated Home Environments13 Jun 2025
- 2025SPHERE: An Evaluation Card for Human-AI Systems24 Mar 2025
- 2025Understanding the LLM-ification of CHI: Unpacking the Impact of LLMs at CHI through a Systematic Literature Review22 Jan 2025
- 2024Cocoa: Co-Planning and Co-Execution with AI Agents14 Dec 2024
- 2024AI2T: Building Trustable AI Tutors by Interactively Teaching a Self-Aware Learning Agent26 Nov 2024
- 2024GPT for Games: An Updated Scoping Review (2020-2024)1 Nov 2024
2 more
- 2025When Should Users Check? Modeling Confirmation Frequency in Multi-Step Agentic AI Taskscited by 6
- 2024AI2T: Building Trustable AI Tutors by Interactively Teaching a Self-Aware Learning Agentcited by 5
- 2025AgentSense: Virtual Sensor Data Generation Using LLM Agents in Simulated Home Environmentscited by 4
- 2025OrchVis: Hierarchical Multi-Agent Orchestration for Human Oversightcited by 0
- 2025OrchVis: Hierarchical Multi-Agent Orchestration for Human Oversight28 Oct 2025
- 2025When Should Users Check? Modeling Confirmation Frequency in Multi-Step Agentic AI Tasks6 Oct 2025
- 2025AgentSense: Virtual Sensor Data Generation Using LLM Agents in Simulated Home Environments13 Jun 2025
- 2024AI2T: Building Trustable AI Tutors by Interactively Teaching a Self-Aware Learning Agent26 Nov 2024
- 2024Cocoa: Co-Planning and Co-Execution with AI Agentscited by 38
- 2024Large Language Models and Video Games: A Preliminary Scoping Reviewcited by 23
- 2025Understanding the LLM-ification of CHI: Unpacking the Impact of LLMs at CHI through a Systematic Literature Reviewcited by 16
- 2024GPT for Games: An Updated Scoping Review (2020-2024)cited by 10
- 2025SPHERE: An Evaluation Card for Human-AI Systemscited by 9
- 2025Interactive Program Synthesis for Modeling Collaborative Physical Activities from Narrated Demonstrationscited by 2
- 2024STAND: Self-Aware Precondition Induction for Interactive Task Learningcited by 1
- 2025Online Learning of HTN Methods for integrated LLM-HTN Planningcited by 1
- 2025Challenges in Grounding Language in the Real Worldcited by 0
- 2025Using Natural Language for Human-Robot Collaboration in the Real Worldcited by 0
- 2025Online Learning of HTN Methods for integrated LLM-HTN Planning17 Nov 2025
- 2025Interactive Program Synthesis for Modeling Collaborative Physical Activities from Narrated Demonstrations29 Sep 2025
- 2025Using Natural Language for Human-Robot Collaboration in the Real World15 Aug 2025
- 2025Challenges in Grounding Language in the Real World20 Jun 2025
- 2025SPHERE: An Evaluation Card for Human-AI Systems24 Mar 2025
- 2025Understanding the LLM-ification of CHI: Unpacking the Impact of LLMs at CHI through a Systematic Literature Review22 Jan 2025
- 2024Cocoa: Co-Planning and Co-Execution with AI Agents14 Dec 2024
- 2024GPT for Games: An Updated Scoping Review (2020-2024)1 Nov 2024
- 2024STAND: Self-Aware Precondition Induction for Interactive Task Learning11 Sep 2024
- 2024Large Language Models and Video Games: A Preliminary Scoping Review5 Mar 2024