# Improving Natural Language Interaction with Robots Using Advice

arXiv: https://arxiv.org/abs/1905.04655 · HTML: https://mlsanity.com/paper/1905.04655/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 May 2019.

## Built on by (most cited first, 9 of 9)

- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [REFINER: Reasoning Feedback on Intermediate Representations](https://mlsanity.com/paper/2304.01904/) (2023)
- [Bridging the Gap: A Survey on Integrating (Human) Feedback for Natural Language Generation](https://mlsanity.com/paper/2305.00955/) (2023)
- [Learning to Repair: Repairing model output errors after deployment using a dynamic memory of feedback](https://mlsanity.com/paper/2112.09737/) (2021)
- [Improving Grounded Language Understanding in a Collaborative Environment by Interacting with Agents Through Help Feedback](https://mlsanity.com/paper/2304.10750/) (2023)
- [Interscript: A dataset for interactive learning of scripts through error feedback](https://mlsanity.com/paper/2112.07867/) (2021)
- [Improving Neural Model Performance through Natural Language Feedback on Their Explanations](https://mlsanity.com/paper/2104.08765/) (2021)
- [Repairs in a Block World: A New Benchmark for Handling User Corrections with Multi-Modal Language Models](https://mlsanity.com/paper/2409.14247/) (2024)
- [IDAT: A Multi-Modal Dataset and Toolkit for Building and Evaluating Interactive Task-Solving Agents](https://mlsanity.com/paper/2407.08898/) (2024)

## Abstract (arXiv)

Over the last few years, there has been growing interest in learning models for physically grounded language understanding tasks, such as the popular blocks world domain. These works typically view this problem as a single-step process, in which a human operator gives an instruction and an automated agent is evaluated on its ability to execute it. In this paper we take the first step towards increasing the bandwidth of this interaction, and suggest a protocol for including advice, high-level observations about the task, which can help constrain the agent's prediction. We evaluate our approach on the blocks world task, and show that even simple advice can help lead to significant performance improvements. To help reduce the effort involved in supplying the advice, we also explore model self-generated advice which can still improve results.