# Connecting First and Second Order Recurrent Networks with Deterministic Finite Automata

arXiv: https://arxiv.org/abs/1911.04644 · HTML: https://mlsanity.com/paper/1911.04644/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Nov 2019.

## Built on by (most cited first, 1 of 1)

- [Synthesizing Context-free Grammars from Recurrent Neural Networks (Extended Version)](https://mlsanity.com/paper/2101.08200/) (2021)

## Abstract (arXiv)

We propose an approach that connects recurrent networks with different orders of hidden interaction with regular grammars of different levels of complexity. We argue that the correspondence between recurrent networks and formal computational models gives understanding to the analysis of the complicated behaviors of recurrent networks. We introduce an entropy value that categorizes all regular grammars into three classes with different levels of complexity, and show that several existing recurrent networks match grammars from either all or partial classes. As such, the differences between regular grammars reveal the different properties of these models. We also provide a unification of all investigated recurrent networks. Our evaluation shows that the unified recurrent network has improved performance in learning grammars, and demonstrates comparable performance on a real-world dataset with more complicated models.