# Guiding Symbolic Natural Language Grammar Induction via Transformer-Based Sequence Probabilities

arXiv: https://arxiv.org/abs/2005.12533 · HTML: https://mlsanity.com/paper/2005.12533/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 May 2020.

## Built on by (most cited first, 2 of 2)

- [The General Theory of General Intelligence: A Pragmatic Patternist Perspective](https://mlsanity.com/paper/2103.15100/) (2021)
- [A Neural Model for Regular Grammar Induction](https://mlsanity.com/paper/2209.11628/) (2022)

## Abstract (arXiv)

A novel approach to automated learning of syntactic rules governing natural languages is proposed, based on using probabilities assigned to sentences (and potentially longer word sequences) by transformer neural network language models to guide symbolic learning processes like clustering and rule induction. This method exploits the learned linguistic knowledge in transformers, without any reference to their inner representations; hence, the technique is readily adaptable to the continuous appearance of more powerful language models. We show a proof-of-concept example of our proposed technique, using it to guide unsupervised symbolic link-grammar induction methods drawn from our prior research.