# "How to best say it?" : Translating Directives in Machine Language into Natural Language in the Blocks World

arXiv: https://arxiv.org/abs/2107.06886 · HTML: https://mlsanity.com/paper/2107.06886/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 2 of its 23 references that are in the map.
- First posted on arXiv 14 Jul 2021.

## Builds on (most cited first, 2 of 2)

- [Hallmarks of Human-Machine Collaboration: A framework for assessment in the DARPA Communicating with Computers Program](https://mlsanity.com/paper/2102.04958/) (2021)
- [Composition by Conversation](https://mlsanity.com/paper/1709.02076/) (2017)

## Abstract (arXiv)

We propose a method to generate optimal natural language for block placement directives generated by a machine's planner during human-agent interactions in the blocks world. A non user-friendly machine directive, e.g., move(ObjId, toPos), is transformed into visually and contextually grounded referring expressions that are much easier for the user to comprehend. We describe an algorithm that progressively and generatively transforms the machine's directive in ECI (Elementary Composable Ideas)-space, generating many alternative versions of the directive. We then define a cost function to evaluate the ease of comprehension of these alternatives and select the best option. The parameters for this cost function were derived empirically from a user study that measured utterance-to-action timings.