# How Should Agents Ask Questions For Situated Learning? An Annotated Dialogue Corpus

arXiv: https://arxiv.org/abs/2106.06504 · HTML: https://mlsanity.com/paper/2106.06504/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 29 references that are in the map.
- First posted on arXiv 11 Jun 2021.

## Builds on (most cited first, 2 of 2)

- [Vision-and-Dialog Navigation](https://mlsanity.com/paper/1907.04957/) (2019)
- [Investigating Evaluation of Open-Domain Dialogue Systems With Human Generated Multiple References](https://mlsanity.com/paper/1907.10568/) (2019)

## Built on by (most cited first, 11 of 11)

- [Asking the Right Question at the Right Time: Human and Model Uncertainty Guidance to Ask Clarification Questions](https://mlsanity.com/paper/2402.06509/) (2024)
- [Dialog Acts for Task-Driven Embodied Agents](https://mlsanity.com/paper/2209.12953/) (2022)
- [Instruction Clarification Requests in Multimodal Collaborative Dialogue Games: Tasks, and an Analysis of the CoDraw Dataset](https://mlsanity.com/paper/2302.14406/) (2023)
- [Simulating User Agents for Embodied Conversational-AI](https://mlsanity.com/paper/2410.23535/) (2024)
- [Referential ambiguity and clarification requests: comparing human and LLM behaviour](https://mlsanity.com/paper/2507.10445/) (2025)
- [Taking Action Towards Graceful Interaction: The Effects of Performing Actions on Modelling Policies for Instruction Clarification Requests](https://mlsanity.com/paper/2401.17039/) (2024)
- [SCOUT: A Situated and Multi-Modal Human-Robot Dialogue Corpus](https://mlsanity.com/paper/2411.12844/) (2024)
- [Decision-Theoretic Question Generation for Situated Reference Resolution: An Empirical Study and Computational Model](https://mlsanity.com/paper/2110.06288/) (2021)
- ["Are you telling me to put glasses on the dog?'' Content-Grounded Annotation of Instruction Clarification Requests in the CoDraw Dataset](https://mlsanity.com/paper/2306.02377/) (2023)
- [Interaction is all You Need? A Study of Robots Ability to Understand and Execute](https://mlsanity.com/paper/2311.07150/) (2023)
- [Navigating to Success in Multi-Modal Human-Robot Collaboration: Analysis and Corpus Release](https://mlsanity.com/paper/2310.17568/) (2023)

## Abstract (arXiv)

Intelligent agents that are confronted with novel concepts in situated environments will need to ask their human teammates questions to learn about the physical world. To better understand this problem, we need data about asking questions in situated task-based interactions. To this end, we present the Human-Robot Dialogue Learning (HuRDL) Corpus - a novel dialogue corpus collected in an online interactive virtual environment in which human participants play the role of a robot performing a collaborative tool-organization task. We describe the corpus data and a corresponding annotation scheme to offer insight into the form and content of questions that humans ask to facilitate learning in a situated environment. We provide the corpus as an empirically-grounded resource for improving question generation in situated intelligent agents.