# The Image Reconstruction Game: Drawing Common Ground Through Iterative Multimodal Dialogue

arXiv: https://arxiv.org/abs/2606.01901 · HTML: https://mlsanity.com/paper/2606.01901/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 28 of its 48 references that are in the map.
- First posted on arXiv 1 Jun 2026.

## Placed in

- Multimodal LLMs / vision-language models / [VLM benchmarks and visual reasoning](https://mlsanity.com/mllm/vlm-benchmarks-and-visual-reasoning/)

## Builds on (most cited first, 28 of 28)

- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI](https://mlsanity.com/paper/2311.16502/) (2023)
- [ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks](https://mlsanity.com/paper/1912.01734/) (2019)
- [LLMScore: Unveiling the Power of Large Language Models in Text-to-Image Synthesis Evaluation](https://mlsanity.com/paper/2305.11116/) (2023)
- [PromptCharm: Text-to-Image Generation through Multi-modal Prompting and Refinement](https://mlsanity.com/paper/2403.04014/) (2024)
- [Tell, Draw, and Repeat: Generating and Modifying Images Based on Continual Linguistic Instruction](https://mlsanity.com/paper/1811.09845/) (2018)
- [Clembench: Using Game Play to Evaluate Chat-Optimized Language Models as Conversational Agents](https://mlsanity.com/paper/2305.13455/) (2023)
- [Large language models predict human sensory judgments across six modalities](https://mlsanity.com/paper/2302.01308/) (2023)
- [The PhotoBook Dataset: Building Common Ground through Visually-Grounded Dialogue](https://mlsanity.com/paper/1906.01530/) (2019)
- [Asking the Right Question at the Right Time: Human and Model Uncertainty Guidance to Ask Clarification Questions](https://mlsanity.com/paper/2402.06509/) (2024)
- [SketchAgent: Language-Driven Sequential Sketch Generation](https://mlsanity.com/paper/2411.17673/) (2024)
- [Learning to Draw: Emergent Communication through Sketching](https://mlsanity.com/paper/2106.02067/) (2021)
- [Math Blind: Failures in Diagram Understanding Undermine Reasoning in MLLMs](https://mlsanity.com/paper/2503.20745/) (2025)
- [LipSim: A Provably Robust Perceptual Similarity Metric](https://mlsanity.com/paper/2310.18274/) (2023)
- [Talk2Image: A Multi-Agent System for Multi-Turn Image Generation and Editing](https://mlsanity.com/paper/2508.06916/) (2025)
- [Instruction Clarification Requests in Multimodal Collaborative Dialogue Games: Tasks, and an Analysis of the CoDraw Dataset](https://mlsanity.com/paper/2302.14406/) (2023)
- [Twin Co-Adaptive Dialogue for Progressive Image Generation](https://mlsanity.com/paper/2504.14868/) (2025)
- [A Collaborative, Interactive and Context-Aware Drawing Agent for Co-Creative Design](https://mlsanity.com/paper/2209.12588/) (2022)
- [MUG: Interactive Multimodal Grounding on User Interfaces](https://mlsanity.com/paper/2209.15099/) (2022)
- [DialogPaint: A Dialog-based Image Editing Model](https://mlsanity.com/paper/2303.10073/) (2023)
- [Scones: Towards Conversational Authoring of Sketches](https://mlsanity.com/paper/2005.07781/) (2020)
- [Iconary: A Pictionary-Based Game for Testing Multimodal Communication with Drawings and Text](https://mlsanity.com/paper/2112.00800/) (2021)
- [GGBench: A Geometric Generative Reasoning Benchmark for Unified Multimodal Models](https://mlsanity.com/paper/2511.11134/) (2025)
- [From Language Games to Drawing Games](https://mlsanity.com/paper/2010.02820/) (2020)
- [MathOPEval: A Fine-grained Evaluation Benchmark for Visual Operations of MLLMs in Mathematical Reasoning](https://mlsanity.com/paper/2507.18140/) (2025)
- [AI Drawing Partner: Co-Creative Drawing Agent and Research Platform to Model Co-Creation](https://mlsanity.com/paper/2501.06607/) (2025)
- [TalkPhoto: A Versatile Training-Free Conversational Assistant for Intelligent Image Editing](https://mlsanity.com/paper/2601.01915/) (2026)
- [TDRI: Two-Phase Dialogue Refinement and Co-Adaptation for Interactive Image Generation](https://mlsanity.com/paper/2503.17669/) (2025)

## Abstract (arXiv)

We introduce the Image Reconstruction Game, a fully automated benchmark in which a vision-language model issues corrective instructions to an image generator across multiple turns, making accumulated common ground directly observable as a rendered image. Benchmarking two Describer models crossed with two Generator models across seven image categories, we find that the describer is the dominant factor in reconstruction quality, while the generator determines whether iterative refinement helps or hurts. Mathematical and geometric images pose the greatest challenge. The describer's token budget strongly affects convergence: shorter budgets yield sparser first renderings with more room for visible improvement, while longer budgets raise absolute quality but leave less to fix. Stronger describers use a richer correction vocabulary spanning spatial, numeric, and structural categories, while weaker describers concentrate on surface properties and tend to stop after a few turns. Human validation shows that the best automated judge reaches only slight-to-fair agreement with human preferences, and automated scores require human recalibration to be used reliably.