# Iris: A Conversational Agent for Complex Tasks

arXiv: https://arxiv.org/abs/1707.05015 · HTML: https://mlsanity.com/paper/1707.05015/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jul 2017.

## Built on by (most cited first, 22 of 22)

- [Graphologue: Exploring Large Language Model Responses with Interactive Diagrams](https://mlsanity.com/paper/2305.11473/) (2023)
- [ConvXAI: Delivering Heterogeneous AI Explanations via Conversations to Support Human-AI Scientific Writing](https://mlsanity.com/paper/2305.09770/) (2023)
- [Interactive Grounded Language Understanding in a Collaborative Environment: IGLU 2021](https://mlsanity.com/paper/2205.02388/) (2022)
- [Saga: A Platform for Continuous Construction and Serving of Knowledge At Scale](https://mlsanity.com/paper/2204.07309/) (2022)
- [XNLI: Explaining and Diagnosing NLI-based Visual Data Analysis](https://mlsanity.com/paper/2301.10385/) (2023)
- [NeurIPS 2021 Competition IGLU: Interactive Grounded Language Understanding in a Collaborative Environment](https://mlsanity.com/paper/2110.06536/) (2021)
- [Transforming Human-Centered AI Collaboration: Redefining Embodied Agents Capabilities through Interactive Grounded Language Instructions](https://mlsanity.com/paper/2305.10783/) (2023)
- [Conversational AI Threads for Visualizing Multidimensional Datasets](https://mlsanity.com/paper/2311.05590/) (2023)
- [Rewriting the Script: Adapting Text Instructions for Voice Interaction](https://mlsanity.com/paper/2306.09992/) (2023)
- [SimVecVis: A Dataset for Enhancing MLLMs in Visualization Understanding](https://mlsanity.com/paper/2506.21319/) (2025)
- [Sketch-based Creativity Support Tools using Deep Learning](https://mlsanity.com/paper/2111.09991/) (2021)
- [Digital Modeling for Everyone: Exploring How Novices Approach Voice-Based 3D Modeling](https://mlsanity.com/paper/2307.04481/) (2023)
- [Learning a Hierarchical Planner from Humans in Multiple Generations](https://mlsanity.com/paper/2310.11614/) (2023)
- [IDAT: A Multi-Modal Dataset and Toolkit for Building and Evaluating Interactive Task-Solving Agents](https://mlsanity.com/paper/2407.08898/) (2024)
- [Visual Intelligence through Human Interaction](https://mlsanity.com/paper/2111.06913/) (2021)
- [A Reference Model for Collaborative Business Intelligence Virtual Assistants](https://mlsanity.com/paper/2304.10556/) (2023)
- [Enhancing Virtual Assistant Intelligence: Precise Area Targeting for Instance-level User Intents beyond Metadata](https://mlsanity.com/paper/2306.04163/) (2023)
- [PROMISE: A Framework for Developing Complex Conversational Interactions (Technical Report)](https://mlsanity.com/paper/2312.03699/) (2023)
- [Survey for Categorising Explainable AI Studies Using Data Analysis Task Frameworks](https://mlsanity.com/paper/2507.10208/) (2025)
- [Bridging Natural Language and Interactive What-If Interfaces via LLM-Generated Declarative Specifications](https://mlsanity.com/paper/2604.07652/) (2026)
- [Q&A or Document-Based? The Effects of Interface Type on How Screen Reader Users Access Interconnected Documents](https://mlsanity.com/paper/2608.25382/) (2026)
- [Richard: Voice-First Mobile Interaction for Persistent Tasks](https://mlsanity.com/paper/2609.39976/) (2026)

## Abstract (arXiv)

Today's conversational agents are restricted to simple standalone commands. In this paper, we present Iris, an agent that draws on human conversational strategies to combine commands, allowing it to perform more complex tasks that it has not been explicitly designed to support: for example, composing one command to "plot a histogram" with another to first "log-transform the data". To enable this complexity, we introduce a domain specific language that transforms commands into automata that Iris can compose, sequence, and execute dynamically by interacting with a user through natural language, as well as a conversational type system that manages what kinds of commands can be combined. We have designed Iris to help users with data science tasks, a domain that requires support for command combination. In evaluation, we find that data scientists complete a predictive modeling task significantly faster (2.6 times speedup) with Iris than a modern non-conversational programming environment. Iris supports the same kinds of commands as today's agents, but empowers users to weave together these commands to accomplish complex goals.