# Improved Latent Tree Induction with Distant Supervision via Span Constraints

arXiv: https://arxiv.org/abs/2109.05112 · HTML: https://mlsanity.com/paper/2109.05112/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 70 references that are in the map.
- First posted on arXiv 10 Sep 2021.

## Builds on (most cited first, 24 of 24)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Adversarial Example Generation with Syntactically Controlled Paraphrase Networks](https://mlsanity.com/paper/1804.06059/) (2018)
- [Sentence Encoders on STILTs: Supplementary Training on Intermediate Labeled-data Tasks](https://mlsanity.com/paper/1811.01088/) (2018)
- [Constituency Parsing with a Self-Attentive Encoder](https://mlsanity.com/paper/1805.01052/) (2018)
- [Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks](https://mlsanity.com/paper/1810.09536/) (2018)
- [MedMentions: A Large Biomedical Corpus Annotated with UMLS Concepts](https://mlsanity.com/paper/1902.09476/) (2019)
- [Compound Probabilistic Context-Free Grammars for Grammar Induction](https://mlsanity.com/paper/1906.10225/) (2019)
- [Unsupervised Latent Tree Induction with Deep Inside-Outside Recursive Autoencoders](https://mlsanity.com/paper/1904.02142/) (2019)
- [A Minimal Span-Based Neural Constituency Parser](https://mlsanity.com/paper/1705.03919/) (2017)
- [Neural Language Modeling by Jointly Learning Syntax and Lexicon](https://mlsanity.com/paper/1711.02013/) (2017)
- [Visually Grounded Neural Syntax Acquisition](https://mlsanity.com/paper/1906.02890/) (2019)
- [Unsupervised Recurrent Neural Network Grammars](https://mlsanity.com/paper/1904.03746/) (2019)
- [Improving AMR Parsing with Sequence-to-Sequence Pre-training](https://mlsanity.com/paper/2010.01771/) (2020)
- [Syntactic Scaffolds for Semantic Structures](https://mlsanity.com/paper/1808.10485/) (2018)
- [Named Entity Recognition with Partially Annotated Training Data](https://mlsanity.com/paper/1909.09270/) (2019)
- [Unsupervised Parsing via Constituency Tests](https://mlsanity.com/paper/2010.03146/) (2020)
- [Soft Gazetteers for Low-Resource Named Entity Recognition](https://mlsanity.com/paper/2005.01866/) (2020)
- [Syntactically Supervised Transformers for Faster Neural Machine Translation](https://mlsanity.com/paper/1906.02780/) (2019)
- [Syntactic Structure Distillation Pretraining For Bidirectional Encoders](https://mlsanity.com/paper/2005.13482/) (2020)
- [LP-SparseMAP: Differentiable Relaxed Optimization for Sparse Structured Prediction](https://mlsanity.com/paper/2001.04437/) (2020)
- [Unsupervised Grammar Induction with Depth-bounded PCFG](https://mlsanity.com/paper/1802.08545/) (2018)
- [Towards Semi-Supervised Learning for Deep Semantic Role Labeling](https://mlsanity.com/paper/1808.09543/) (2018)
- [Learning Syntax from Naturally-Occurring Bracketings](https://mlsanity.com/paper/2104.13933/) (2021)

## Built on by (most cited first, 6 of 6)

- [Dynamic Programming in Rank Space: Scaling Structured Inference with Low-Rank HMMs and PCFGs](https://mlsanity.com/paper/2205.00484/) (2022)
- [Improve Event Extraction via Self-Training with Gradient Guidance](https://mlsanity.com/paper/2205.12490/) (2022)
- [Forming Trees with Treeformers](https://mlsanity.com/paper/2207.06960/) (2022)
- [Simple Hardware-Efficient PCFGs with Independent Left and Right Productions](https://mlsanity.com/paper/2310.14997/) (2023)
- [Learning with Latent Structures in Natural Language Processing: A Survey](https://mlsanity.com/paper/2201.00490/) (2022)
- [Improving Unsupervised Constituency Parsing via Maximizing Semantic Information](https://mlsanity.com/paper/2410.02558/) (2024)

## Abstract (arXiv)

For over thirty years, researchers have developed and analyzed methods for latent tree induction as an approach for unsupervised syntactic parsing. Nonetheless, modern systems still do not perform well enough compared to their supervised counterparts to have any practical use as structural annotation of text. In this work, we present a technique that uses distant supervision in the form of span constraints (i.e. phrase bracketing) to improve performance in unsupervised constituency parsing. Using a relatively small number of span constraints we can substantially improve the output from DIORA, an already competitive unsupervised parsing system. Compared with full parse tree annotation, span constraints can be acquired with minimal effort, such as with a lexicon derived from Wikipedia, to find exact text matches. Our experiments show span constraints based on entities improves constituency parsing on English WSJ Penn Treebank by more than 5 F1. Furthermore, our method extends to any domain where span constraints are easily attainable, and as a case study we demonstrate its effectiveness by parsing biomedical text from the CRAFT dataset.