# Recursive Decoding: A Situated Cognition Approach to Compositional Generation in Grounded Language Understanding

arXiv: https://arxiv.org/abs/2201.11766 · HTML: https://mlsanity.com/paper/2201.11766/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 34 references that are in the map.
- First posted on arXiv 27 Jan 2022.

## Builds on (most cited first, 22 of 22)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering](https://mlsanity.com/paper/1612.00837/) (2016)
- [Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environments](https://mlsanity.com/paper/1711.07280/) (2017)
- [ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks](https://mlsanity.com/paper/1912.01734/) (2019)
- [Compositionality decomposed: how do neural networks generalise?](https://mlsanity.com/paper/1908.08351/) (2019)
- [COGS: A Compositional Generalization Challenge Based on Semantic Interpretation](https://mlsanity.com/paper/2010.05465/) (2020)
- [Self-Monitoring Navigation Agent via Auxiliary Progress Estimation](https://mlsanity.com/paper/1901.03035/) (2019)
- [Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval](https://mlsanity.com/paper/2009.12756/) (2020)
- [Good-Enough Compositional Data Augmentation](https://mlsanity.com/paper/1904.09545/) (2019)
- [A Benchmark for Systematic Generalization in Grounded Language Understanding](https://mlsanity.com/paper/2003.05161/) (2020)
- [Compositional generalization through meta sequence-to-sequence learning](https://mlsanity.com/paper/1906.05381/) (2019)
- [Learning to Recombine and Resample Data for Compositional Generalization](https://mlsanity.com/paper/2010.03706/) (2020)
- [The paradox of the compositionality of natural language: a neural machine translation case study](https://mlsanity.com/paper/2108.05885/) (2021)
- [Human few-shot learning of compositional instructions](https://mlsanity.com/paper/1901.04587/) (2019)
- [The EOS Decision and Length Extrapolation](https://mlsanity.com/paper/2010.07174/) (2020)
- [Disentangled Sequence to Sequence Learning for Compositional Generalization](https://mlsanity.com/paper/2110.04655/) (2021)
- [Compositional Networks Enable Systematic Generalization for Grounded Language Understanding](https://mlsanity.com/paper/2008.02742/) (2020)
- [Systematic Generalization on gSCAN with Language Conditioned Embedding](https://mlsanity.com/paper/2009.05552/) (2020)
- [Think before you act: A simple baseline for compositional generalization](https://mlsanity.com/paper/2009.13962/) (2020)
- [Iterative Decoding for Compositional Generalization in Transformers](https://mlsanity.com/paper/2110.04169/) (2021)
- [On the Realization of Compositionality in Neural Networks](https://mlsanity.com/paper/1906.01634/) (2019)

## Built on by (most cited first, 3 of 3)

- [Improved Compositional Generalization by Generating Demonstrations for Meta-Learning](https://mlsanity.com/paper/2305.13092/) (2023)
- [A Hybrid System for Systematic Generalization in Simple Arithmetic Problems](https://mlsanity.com/paper/2306.17249/) (2023)
- [Learning neuro-symbolic convergent term rewriting systems](https://mlsanity.com/paper/2507.19372/) (2025)

## Abstract (arXiv)

Compositional generalization is a troubling blind spot for neural language models. Recent efforts have presented techniques for improving a model's ability to encode novel combinations of known inputs, but less work has focused on generating novel combinations of known outputs. Here we focus on this latter "decode-side" form of generalization in the context of gSCAN, a synthetic benchmark for compositional generalization in grounded language understanding. We present Recursive Decoding (RD), a novel procedure for training and using seq2seq models, targeted towards decode-side generalization. Rather than generating an entire output sequence in one pass, models are trained to predict one token at a time. Inputs (i.e., the external gSCAN environment) are then incrementally updated based on predicted tokens, and re-encoded for the next decoder time step. RD thus decomposes a complex, out-of-distribution sequence generation task into a series of incremental predictions that each resemble what the model has already seen during training. RD yields dramatic improvement on two previously neglected generalization tasks in gSCAN. We provide analyses to elucidate these gains over failure of a baseline, and then discuss implications for generalization in naturalistic grounded language understanding, and seq2seq more generally.