# Fast and Accurate Neural CRF Constituency Parsing

arXiv: https://arxiv.org/abs/2008.03736 · HTML: https://mlsanity.com/paper/2008.03736/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Aug 2020.

## Built on by (most cited first, 30 of 50)

- [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://mlsanity.com/paper/2111.01243/) (2021)
- [Grammar-Constrained Decoding for Structured NLP Tasks without Finetuning](https://mlsanity.com/paper/2305.13971/) (2023)
- [Self-Attention Networks Can Process Bounded Hierarchical Languages](https://mlsanity.com/paper/2105.11115/) (2021)
- [Semantic Probabilistic Layers for Neuro-Symbolic Learning](https://mlsanity.com/paper/2206.00426/) (2022)
- [Pushing on Text Readability Assessment: A Transformer Meets Handcrafted Linguistic Features](https://mlsanity.com/paper/2109.12258/) (2021)
- [The Stem Cell Hypothesis: Dilemma behind Multi-Task Learning with Transformer Encoders](https://mlsanity.com/paper/2109.06939/) (2021)
- [PCFGs Can Do Better: Inducing Probabilistic Context-Free Grammars with Many Symbols](https://mlsanity.com/paper/2104.13727/) (2021)
- [Multi-granularity Textual Adversarial Attack with Behavior Cloning](https://mlsanity.com/paper/2109.04367/) (2021)
- [Bottom-Up Constituency Parsing and Nested Named Entity Recognition with Pointer Networks](https://mlsanity.com/paper/2110.05419/) (2021)
- [Neural Bi-Lexicalized PCFG Induction](https://mlsanity.com/paper/2105.15021/) (2021)
- [Nested Named Entity Recognition as Latent Lexicalized Constituency Parsing](https://mlsanity.com/paper/2203.04665/) (2022)
- [Headed-Span-Based Projective Dependency Parsing](https://mlsanity.com/paper/2108.04750/) (2021)
- [BiSyn-GAT+: Bi-Syntax Aware Graph Attention Network for Aspect-based Sentiment Analysis](https://mlsanity.com/paper/2204.03117/) (2022)
- [Dynamic Programming in Rank Space: Scaling Structured Inference with Low-Rank HMMs and PCFGs](https://mlsanity.com/paper/2205.00484/) (2022)
- [Temporal Knowledge Graph Question Answering: A Survey](https://mlsanity.com/paper/2406.14191/) (2024)
- [A Conditional Splitting Framework for Efficient Constituency Parsing](https://mlsanity.com/paper/2106.15760/) (2021)
- [PiTe: Pixel-Temporal Alignment for Large Video-Language Model](https://mlsanity.com/paper/2409.07239/) (2024)
- [Ensemble Distillation for Unsupervised Constituency Parsing](https://mlsanity.com/paper/2310.01717/) (2023)
- [Data-centric NLP Backdoor Defense from the Lens of Memorization](https://mlsanity.com/paper/2409.14200/) (2024)
- [Have Attention Heads in BERT Learned Constituency Grammar?](https://mlsanity.com/paper/2102.07926/) (2021)
- [Augmenting Transformers with Recursively Composed Multi-grained Representations](https://mlsanity.com/paper/2309.16319/) (2023)
- [Improving BERT Pretraining with Syntactic Supervision](https://mlsanity.com/paper/2104.10516/) (2021)
- [Investigating Non-local Features for Neural Constituency Parsing](https://mlsanity.com/paper/2109.12814/) (2021)
- [Traditional Readability Formulas Compared for English](https://mlsanity.com/paper/2301.02975/) (2023)
- [A Multi-Grained Self-Interpretable Symbolic-Neural Model For Single/Multi-Labeled Text Classification](https://mlsanity.com/paper/2303.02860/) (2023)
- [Hermes: Unlocking Security Analysis of Cellular Network Protocols by Synthesizing Finite State Machines from Natural Language Specifications](https://mlsanity.com/paper/2310.04381/) (2023)
- [Why Don't Prompt-Based Fairness Metrics Correlate?](https://mlsanity.com/paper/2406.05918/) (2024)
- [Recursive Bayesian Networks: Generalising and Unifying Probabilistic Context-Free Grammars and Dynamic Bayesian Networks](https://mlsanity.com/paper/2111.01853/) (2021)
- [Joint Chinese Word Segmentation and Part-of-speech Tagging via Two-stage Span Labeling](https://mlsanity.com/paper/2112.09488/) (2021)
- [ELIT: Emory Language and Information Toolkit](https://mlsanity.com/paper/2109.03903/) (2021)

## Abstract (arXiv)

Estimating probability distribution is one of the core issues in the NLP field. However, in both deep learning (DL) and pre-DL eras, unlike the vast applications of linear-chain CRF in sequence labeling tasks, very few works have applied tree-structure CRF to constituency parsing, mainly due to the complexity and inefficiency of the inside-outside algorithm. This work presents a fast and accurate neural CRF constituency parser. The key idea is to batchify the inside algorithm for loss computation by direct large tensor operations on GPU, and meanwhile avoid the outside algorithm for gradient computation via efficient back-propagation. We also propose a simple two-stage bracketing-then-labeling parsing approach to improve efficiency further. To improve the parsing performance, inspired by recent progress in dependency parsing, we introduce a new scoring architecture based on boundary representation and biaffine attention, and a beneficial dropout strategy. Experiments on PTB, CTB5.1, and CTB7 show that our two-stage CRF parser achieves new state-of-the-art performance on both settings of w/o and w/ BERT, and can parse over 1,000 sentences per second. We release our code at https://github.com/yzhangcs/crfpar.