# Unsupervised Latent Tree Induction with Deep Inside-Outside Recursive Autoencoders

arXiv: https://arxiv.org/abs/1904.02142 · HTML: https://mlsanity.com/paper/1904.02142/

## Key facts

- Cited by 60 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Apr 2019.

## Built on by (most cited first, 30 of 60)

- [LasUIE: Unifying Information Extraction with Latent Adaptive Structure-aware Generative Language Model](https://mlsanity.com/paper/2304.06248/) (2023)
- [VLGrammar: Grounded Grammar Induction of Vision and Language](https://mlsanity.com/paper/2103.12975/) (2021)
- [PCFGs Can Do Better: Inducing Probabilistic Context-Free Grammars with Many Symbols](https://mlsanity.com/paper/2104.13727/) (2021)
- [R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling](https://mlsanity.com/paper/2107.00967/) (2021)
- [Interpreting Deep Learning Models in Natural Language Processing: A Review](https://mlsanity.com/paper/2110.10470/) (2021)
- [Pushdown Layers: Encoding Recursive Structure in Transformer Language Models](https://mlsanity.com/paper/2310.19089/) (2023)
- [Neural Bi-Lexicalized PCFG Induction](https://mlsanity.com/paper/2105.15021/) (2021)
- [Video-aided Unsupervised Grammar Induction](https://mlsanity.com/paper/2104.04369/) (2021)
- [Modeling Hierarchical Structures with Continuous Recursive Neural Networks](https://mlsanity.com/paper/2106.06038/) (2021)
- [Fast-R2D2: A Pretrained Recursive Neural Network based on Pruned CKY for Grammar Induction and Text Representation](https://mlsanity.com/paper/2203.00281/) (2022)
- [Grammar-Based Grounded Lexicon Learning](https://mlsanity.com/paper/2202.08806/) (2022)
- [Generative Pretrained Structured Transformers: Unsupervised Syntactic Language Models at Scale](https://mlsanity.com/paper/2403.08293/) (2024)
- [Unsupervised Vision-Language Parsing: Seamlessly Bridging Visual Scene Graphs with Language Structures via Dependency Relationships](https://mlsanity.com/paper/2203.14260/) (2022)
- [Contextual Distortion Reveals Constituency: Masked Language Models are Implicit Parsers](https://mlsanity.com/paper/2306.00645/) (2023)
- [Improved Latent Tree Induction with Distant Supervision via Span Constraints](https://mlsanity.com/paper/2109.05112/) (2021)
- [Beam Tree Recursive Cells](https://mlsanity.com/paper/2305.19999/) (2023)
- [Ensemble Distillation for Unsupervised Constituency Parsing](https://mlsanity.com/paper/2310.01717/) (2023)
- [Augmenting Transformers with Recursively Composed Multi-grained Representations](https://mlsanity.com/paper/2309.16319/) (2023)
- [Recursion in Recursion: Two-Level Nested Recursion for Length Generalization with Scalability](https://mlsanity.com/paper/2311.04449/) (2023)
- [Learning Syntax from Naturally-Occurring Bracketings](https://mlsanity.com/paper/2104.13933/) (2021)
- [Rule Augmented Unsupervised Constituency Parsing](https://mlsanity.com/paper/2105.10193/) (2021)
- [A Multi-Grained Self-Interpretable Symbolic-Neural Model For Single/Multi-Labeled Text Classification](https://mlsanity.com/paper/2303.02860/) (2023)
- [Efficient Beam Tree Recursion](https://mlsanity.com/paper/2307.10779/) (2023)
- [Graph Parsing Networks](https://mlsanity.com/paper/2402.14393/) (2024)
- [DeepCPCFG: Deep Learning and Context Free Grammars for End-to-End Information Extraction](https://mlsanity.com/paper/2103.05908/) (2021)
- [Forming Trees with Treeformers](https://mlsanity.com/paper/2207.06960/) (2022)
- [Learning a Grammar Inducer from Massive Uncurated Instructional Videos](https://mlsanity.com/paper/2210.12309/) (2022)
- [Efficient Semiring-Weighted Earley Parsing](https://mlsanity.com/paper/2307.02982/) (2023)
- [Weakly Supervised Reasoning by Neuro-Symbolic Approaches](https://mlsanity.com/paper/2309.13072/) (2023)
- [ChapGTP, ILLC's Attempt at Raising a BabyLM: Improving Data Efficiency by Automatic Task Formation](https://mlsanity.com/paper/2310.11282/) (2023)

## Abstract (arXiv)

We introduce deep inside-outside recursive autoencoders (DIORA), a fully-unsupervised method for discovering syntax that simultaneously learns representations for constituents within the induced tree. Our approach predicts each word in an input sentence conditioned on the rest of the sentence and uses inside-outside dynamic programming to consider all possible binary trees over the sentence. At test time the CKY algorithm extracts the highest scoring parse. DIORA achieves a new state-of-the-art F1 in unsupervised binary constituency parsing (unlabeled) in two benchmark datasets, WSJ and MultiNLI.