# Attentive Tensor Product Learning

arXiv: https://arxiv.org/abs/1802.07089 · HTML: https://mlsanity.com/paper/1802.07089/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Feb 2018.

## Built on by (most cited first, 6 of 6)

- [Neurocompositional computing: From the Central Paradox of Cognition to a new generation of AI systems](https://mlsanity.com/paper/2205.01128/) (2022)
- [Is Neuro-Symbolic AI Meeting its Promise in Natural Language Processing? A Structured Review](https://mlsanity.com/paper/2202.12205/) (2022)
- [Enriching Transformers with Structured Tensor-Product Representations for Abstractive Summarization](https://mlsanity.com/paper/2106.01317/) (2021)
- [Quantum Natural Language Processing](https://mlsanity.com/paper/2403.19758/) (2024)
- [Quantum Mathematics in Artificial Intelligence](https://mlsanity.com/paper/2101.04255/) (2021)
- [Should Semantic Vector Composition be Explicit? Can it be Linear?](https://mlsanity.com/paper/2104.06555/) (2021)

## Abstract (arXiv)

This paper proposes a new architecture - Attentive Tensor Product Learning (ATPL) - to represent grammatical structures in deep learning models. ATPL is a new architecture to bridge this gap by exploiting Tensor Product Representations (TPR), a structured neural-symbolic model developed in cognitive science, aiming to integrate deep learning with explicit language structures and rules. The key ideas of ATPL are: 1) unsupervised learning of role-unbinding vectors of words via TPR-based deep neural network; 2) employing attention modules to compute TPR; and 3) integration of TPR with typical deep learning architectures including Long Short-Term Memory (LSTM) and Feedforward Neural Network (FFNN). The novelty of our approach lies in its ability to extract the grammatical structure of a sentence by using role-unbinding vectors, which are obtained in an unsupervised manner. This ATPL approach is applied to 1) image captioning, 2) part of speech (POS) tagging, and 3) constituency parsing of a sentence. Experimental results demonstrate the effectiveness of the proposed approach.