# TPR-Attention for Combinatorial Generalization

arXiv: https://arxiv.org/abs/2608.30124 · HTML: https://mlsanity.com/paper/2608.30124/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 11 references that are in the map.
- First posted on arXiv 31 Aug 2026.

## Builds on (most cited first, 6 of 6)

- [Towards Causal Representation Learning](https://mlsanity.com/paper/2102.11107/) (2021)
- [A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part I: Models and Data Transformations](https://mlsanity.com/paper/2111.06077/) (2021)
- [Unsupervised Model Selection for Variational Disentangled Representation Learning](https://mlsanity.com/paper/1905.12614/) (2019)
- [Neurocompositional computing: From the Central Paradox of Cognition to a new generation of AI systems](https://mlsanity.com/paper/2205.01128/) (2022)
- [Compositional Generalization in Unsupervised Compositional Representation Learning: A Study on Disentanglement and Emergent Language](https://mlsanity.com/paper/2210.00482/) (2022)
- [Lost in Latent Space: Disentangled Models and the Challenge of Combinatorial Generalisation](https://mlsanity.com/paper/2204.02283/) (2022)

## Abstract (arXiv)

Systematic generalization remains a significant challenge in deep learning. In particular, combinatorial generalization - generalizing to new configurations of known factors of variation - is effortless for humans but difficult for standard neural architectures that rely on statistical correlations rather than explicit structural representations. We introduce a new architectural component that embeds structured inductive bias into deep learning: an attention mechanism operating over tensor-product representations (TPRs). Through controlled experiments on compositional tasks, we show that this TPR-attention mechanism outperforms existing architectural components in combinatorial generalization. These results highlight the value of integrating explicit compositional structure into neural attention and point toward a promising path for models capable of systematic generalization.