# A Critical Analysis of Biased Parsers in Unsupervised Parsing

arXiv: https://arxiv.org/abs/1909.09428 · HTML: https://mlsanity.com/paper/1909.09428/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Sep 2019.

## Built on by (most cited first, 7 of 7)

- [Square One Bias in NLP: Towards a Multi-Dimensional Exploration of the Research Manifold](https://mlsanity.com/paper/2206.09755/) (2022)
- [Reuse Your Rewards: Reward Model Transfer for Zero-Shot Cross-Lingual Alignment](https://mlsanity.com/paper/2404.12318/) (2024)
- [Uncovering Constraint-Based Behavior in Neural Models via Targeted Fine-Tuning](https://mlsanity.com/paper/2106.01207/) (2021)
- [An Empirical Study on Leveraging Position Embeddings for Target-oriented Opinion Words Extraction](https://mlsanity.com/paper/2109.01238/) (2021)
- [Linear-Time Modeling of Linguistic Structure: An Order-Theoretic Perspective](https://mlsanity.com/paper/2305.15057/) (2023)
- [Generalized Optimal Linear Orders](https://mlsanity.com/paper/2108.10692/) (2021)
- [Unsupervised and Few-shot Parsing from Pretrained Language Models](https://mlsanity.com/paper/2206.04980/) (2022)

## Abstract (arXiv)

A series of recent papers has used a parsing algorithm due to Shen et al. (2018) to recover phrase-structure trees based on proxies for "syntactic depth." These proxy depths are obtained from the representations learned by recurrent language models augmented with mechanisms that encourage the (unsupervised) discovery of hierarchical structure latent in natural language sentences. Using the same parser, we show that proxies derived from a conventional LSTM language model produce trees comparably well to the specialized architectures used in previous work. However, we also provide a detailed analysis of the parsing algorithm, showing (1) that it is incomplete---that is, it can recover only a fraction of possible trees---and (2) that it has a marked bias for right-branching structures which results in inflated performance in right-branching languages like English. Our analysis shows that evaluating with biased parsing algorithms can inflate the apparent structural competence of language models.