Iris: A Conversational Agent for Complex Tasks
Not on the map. Why?
Abstract (arXiv)
Today's conversational agents are restricted to simple standalone commands. In this paper, we present Iris, an agent that draws on human conversational strategies to combine commands, allowing it to perform more complex tasks that it has not been explicitly designed to support: for example, composing one command to "plot a histogram" with another to first "log-transform the data". To enable this complexity, we introduce a domain specific language that transforms commands into automata that Iris can compose, sequence, and execute dynamically by interacting with a user through natural language, as well as a conversational type system that manages what kinds of commands can be combined. We have designed Iris to help users with data science tasks, a domain that requires support for command combination. In evaluation, we find that data scientists complete a predictive modeling task significantly faster (2.6 times speedup) with Iris than a modern non-conversational programming environment. Iris supports the same kinds of commands as today's agents, but empowers users to weave together these commands to accomplish complex goals.
Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). First posted on arXiv 17 Jul 2017.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 3; 2022: 2; 2023: 11; 2024: 1; 2025: 2; 2026: 3.
22 papers.
Sort:
By area:
- 2023Graphologue: Exploring Large Language Model Responses with Interactive Diagramscited by 65
- 2023ConvXAI: Delivering Heterogeneous AI Explanations via Conversations to Support Human-AI Scientific Writingcited by 31
- 2022Interactive Grounded Language Understanding in a Collaborative Environment: IGLU 2021cited by 23
- 2022Saga: A Platform for Continuous Construction and Serving of Knowledge At Scalecited by 18
- 2023XNLI: Explaining and Diagnosing NLI-based Visual Data Analysiscited by 18
- 2021NeurIPS 2021 Competition IGLU: Interactive Grounded Language Understanding in a Collaborative Environmentcited by 15
- 2023Transforming Human-Centered AI Collaboration: Redefining Embodied Agents Capabilities through Interactive Grounded Language Instructionscited by 13
- 2023Conversational AI Threads for Visualizing Multidimensional Datasetscited by 8
- 2023Rewriting the Script: Adapting Text Instructions for Voice Interactioncited by 5
- 2025SimVecVis: A Dataset for Enhancing MLLMs in Visualization Understandingcited by 5
- 2021Sketch-based Creativity Support Tools using Deep Learningcited by 3
- 2023Digital Modeling for Everyone: Exploring How Novices Approach Voice-Based 3D Modelingcited by 1
10 more
- 2023Learning a Hierarchical Planner from Humans in Multiple Generationscited by 1
- 2024IDAT: A Multi-Modal Dataset and Toolkit for Building and Evaluating Interactive Task-Solving Agentscited by 1
- 2021Visual Intelligence through Human Interactioncited by 0
- 2023A Reference Model for Collaborative Business Intelligence Virtual Assistantscited by 0
- 2023Enhancing Virtual Assistant Intelligence: Precise Area Targeting for Instance-level User Intents beyond Metadatacited by 0
- 2023PROMISE: A Framework for Developing Complex Conversational Interactions (Technical Report)cited by 0
- 2025Survey for Categorising Explainable AI Studies Using Data Analysis Task Frameworkscited by 0
- 2026Bridging Natural Language and Interactive What-If Interfaces via LLM-Generated Declarative Specificationscited by 0
- 2026Q&A or Document-Based? The Effects of Interface Type on How Screen Reader Users Access Interconnected Documentscited by 0
- 2026Richard: Voice-First Mobile Interaction for Persistent Taskscited by 0
- 2026Richard: Voice-First Mobile Interaction for Persistent Tasks30 Sep 2026
- 2026Q&A or Document-Based? The Effects of Interface Type on How Screen Reader Users Access Interconnected Documents26 Aug 2026
- 2026Bridging Natural Language and Interactive What-If Interfaces via LLM-Generated Declarative Specifications8 Apr 2026
- 2025Survey for Categorising Explainable AI Studies Using Data Analysis Task Frameworks14 Jul 2025
- 2025SimVecVis: A Dataset for Enhancing MLLMs in Visualization Understanding26 Jun 2025
- 2024IDAT: A Multi-Modal Dataset and Toolkit for Building and Evaluating Interactive Task-Solving Agents12 Jul 2024
- 2023PROMISE: A Framework for Developing Complex Conversational Interactions (Technical Report)6 Dec 2023
- 2023Conversational AI Threads for Visualizing Multidimensional Datasets9 Nov 2023
- 2023Learning a Hierarchical Planner from Humans in Multiple Generations17 Oct 2023
- 2023Digital Modeling for Everyone: Exploring How Novices Approach Voice-Based 3D Modeling10 Jul 2023
- 2023Rewriting the Script: Adapting Text Instructions for Voice Interaction16 Jun 2023
- 2023Enhancing Virtual Assistant Intelligence: Precise Area Targeting for Instance-level User Intents beyond Metadata7 Jun 2023
10 more
- 2023Graphologue: Exploring Large Language Model Responses with Interactive Diagrams19 May 2023
- 2023Transforming Human-Centered AI Collaboration: Redefining Embodied Agents Capabilities through Interactive Grounded Language Instructions18 May 2023
- 2023ConvXAI: Delivering Heterogeneous AI Explanations via Conversations to Support Human-AI Scientific Writing16 May 2023
- 2023A Reference Model for Collaborative Business Intelligence Virtual Assistants20 Apr 2023
- 2023XNLI: Explaining and Diagnosing NLI-based Visual Data Analysis25 Jan 2023
- 2022Interactive Grounded Language Understanding in a Collaborative Environment: IGLU 20215 May 2022
- 2022Saga: A Platform for Continuous Construction and Serving of Knowledge At Scale15 Apr 2022
- 2021Sketch-based Creativity Support Tools using Deep Learning19 Nov 2021
- 2021Visual Intelligence through Human Interaction12 Nov 2021
- 2021NeurIPS 2021 Competition IGLU: Interactive Grounded Language Understanding in a Collaborative Environment13 Oct 2021
- 2023Graphologue: Exploring Large Language Model Responses with Interactive Diagramscited by 65
- 2023ConvXAI: Delivering Heterogeneous AI Explanations via Conversations to Support Human-AI Scientific Writingcited by 31
- 2022Interactive Grounded Language Understanding in a Collaborative Environment: IGLU 2021cited by 23
- 2022Saga: A Platform for Continuous Construction and Serving of Knowledge At Scalecited by 18
- 2023XNLI: Explaining and Diagnosing NLI-based Visual Data Analysiscited by 18
- 2021NeurIPS 2021 Competition IGLU: Interactive Grounded Language Understanding in a Collaborative Environmentcited by 15
- 2023Transforming Human-Centered AI Collaboration: Redefining Embodied Agents Capabilities through Interactive Grounded Language Instructionscited by 13
- 2023Conversational AI Threads for Visualizing Multidimensional Datasetscited by 8
- 2023Rewriting the Script: Adapting Text Instructions for Voice Interactioncited by 5
- 2021Sketch-based Creativity Support Tools using Deep Learningcited by 3
- 2023Digital Modeling for Everyone: Exploring How Novices Approach Voice-Based 3D Modelingcited by 1
- 2023Learning a Hierarchical Planner from Humans in Multiple Generationscited by 1
12 most cited of 21 All 21 (JSON)
- 2026Richard: Voice-First Mobile Interaction for Persistent Tasks30 Sep 2026
- 2026Q&A or Document-Based? The Effects of Interface Type on How Screen Reader Users Access Interconnected Documents26 Aug 2026
- 2026Bridging Natural Language and Interactive What-If Interfaces via LLM-Generated Declarative Specifications8 Apr 2026
- 2025Survey for Categorising Explainable AI Studies Using Data Analysis Task Frameworks14 Jul 2025
- 2024IDAT: A Multi-Modal Dataset and Toolkit for Building and Evaluating Interactive Task-Solving Agents12 Jul 2024
- 2023PROMISE: A Framework for Developing Complex Conversational Interactions (Technical Report)6 Dec 2023
- 2023Conversational AI Threads for Visualizing Multidimensional Datasets9 Nov 2023
- 2023Learning a Hierarchical Planner from Humans in Multiple Generations17 Oct 2023
- 2023Digital Modeling for Everyone: Exploring How Novices Approach Voice-Based 3D Modeling10 Jul 2023
- 2023Rewriting the Script: Adapting Text Instructions for Voice Interaction16 Jun 2023
- 2023Enhancing Virtual Assistant Intelligence: Precise Area Targeting for Instance-level User Intents beyond Metadata7 Jun 2023
- 2023Graphologue: Exploring Large Language Model Responses with Interactive Diagrams19 May 2023
12 newest of 21 All 21 (JSON)