# Assessing the Unitary RNN as an End-to-End Compositional Model of Syntax

arXiv: https://arxiv.org/abs/2208.05719 · HTML: https://mlsanity.com/paper/2208.05719/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 31 references that are in the map.
- First posted on arXiv 11 Aug 2022.

## Builds on (most cited first, 10 of 10)

- [Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies](https://mlsanity.com/paper/1611.01368/) (2016)
- [Unitary Evolution Recurrent Neural Networks](https://mlsanity.com/paper/1511.06464/) (2015)
- [Colorless green recurrent networks dream hierarchically](https://mlsanity.com/paper/1803.11138/) (2018)
- [Full-Capacity Unitary Recurrent Neural Networks](https://mlsanity.com/paper/1611.00035/) (2016)
- [Tunable Efficient Unitary Neural Networks (EUNN) and their application to RNNs](https://mlsanity.com/paper/1612.05231/) (2016)
- [RNNs can generate bounded hierarchical languages with optimal memory](https://mlsanity.com/paper/2010.07515/) (2020)
- [projUNN: efficient method for training deep networks with unitary matrices](https://mlsanity.com/paper/2203.05483/) (2022)
- [Evaluating the Ability of LSTMs to Learn Context-Free Grammars](https://mlsanity.com/paper/1811.02611/) (2018)
- [Learning Unitary Operators with Help From u(n)](https://mlsanity.com/paper/1607.04903/) (2016)
- [Vector Space Semantics for Lambek Calculus with Soft Subexponentials](https://mlsanity.com/paper/2111.11331/) (2021)

## Built on by (most cited first, 2 of 2)

- [Governance-Aware Hybrid Fine-Tuning for Multilingual Large Language Models](https://mlsanity.com/paper/2512.17344/) (2025)
- [Hybrid and Unitary PEFT for Resource-Efficient Large Language Models](https://mlsanity.com/paper/2507.18076/) (2025)

## Abstract (arXiv)

We show that both an LSTM and a unitary-evolution recurrent neural network (URN) can achieve encouraging accuracy on two types of syntactic patterns: context-free long distance agreement, and mildly context-sensitive cross serial dependencies. This work extends recent experiments on deeply nested context-free long distance dependencies, with similar results. URNs differ from LSTMs in that they avoid non-linear activation functions, and they apply matrix multiplication to word embeddings encoded as unitary matrices. This permits them to retain all information in the processing of an input string over arbitrary distances. It also causes them to satisfy strict compositionality. URNs constitute a significant advance in the search for explainable models in deep learning applied to NLP.