# Modeling Label Correlations for Second-Order Semantic Dependency Parsing with Mean-Field Inference

arXiv: https://arxiv.org/abs/2204.03619 · HTML: https://mlsanity.com/paper/2204.03619/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 63 references that are in the map.
- First posted on arXiv 7 Apr 2022.

## Builds on (most cited first, 25 of 25)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Deep Biaffine Attention for Neural Dependency Parsing](https://mlsanity.com/paper/1611.01734/) (2016)
- [Conditional Random Fields as Recurrent Neural Networks](https://mlsanity.com/paper/1502.03240/) (2015)
- [Efficient Second-Order TreeCRF for Neural Dependency Parsing](https://mlsanity.com/paper/2005.00975/) (2020)
- [Introduction to Tensor Decompositions and their Applications in Machine Learning](https://mlsanity.com/paper/1711.10781/) (2017)
- [Simpler but More Accurate Semantic Dependency Parsing](https://mlsanity.com/paper/1807.01396/) (2018)
- [Broad-Coverage Semantic Parsing as Transduction](https://mlsanity.com/paper/1909.02607/) (2019)
- [PCFGs Can Do Better: Inducing Probabilistic Context-Free Grammars with Many Symbols](https://mlsanity.com/paper/2104.13727/) (2021)
- [Left-to-Right Dependency Parsing with Pointer Networks](https://mlsanity.com/paper/1903.08445/) (2019)
- [Second-Order Semantic Dependency Parsing with End-to-End Neural Networks](https://mlsanity.com/paper/1906.07880/) (2019)
- [Establishing Strong Baselines for the New Decade: Sequence Tagging, Syntactic and Semantic Parsing with BERT](https://mlsanity.com/paper/1908.04943/) (2019)
- [Neural Bi-Lexicalized PCFG Induction](https://mlsanity.com/paper/2105.15021/) (2021)
- [LP-SparseMAP: Differentiable Relaxed Optimization for Sparse Structured Prediction](https://mlsanity.com/paper/2001.04437/) (2020)
- [Second-Order Neural Dependency Parsing with Message Passing and End-to-End Training](https://mlsanity.com/paper/2010.05003/) (2020)
- [Transition-based Semantic Dependency Parsing with Pointer Networks](https://mlsanity.com/paper/2005.13344/) (2020)
- [Compositional Semantic Parsing Across Graphbanks](https://mlsanity.com/paper/1906.11746/) (2019)
- [High-order Semantic Role Labeling](https://mlsanity.com/paper/2010.04641/) (2020)
- [Deep Multitask Learning for Semantic Dependency Parsing](https://mlsanity.com/paper/1704.06855/) (2017)
- [Fast semantic parsing with well-typedness guarantees](https://mlsanity.com/paper/2009.07365/) (2020)
- [Regularized Frank-Wolfe for Dense CRFs: Generalizing Mean Field and Beyond](https://mlsanity.com/paper/2110.14759/) (2021)
- [Multi-Task Semantic Dependency Parsing with Policy Gradient for Learning Easy-First Strategies](https://mlsanity.com/paper/1906.01239/) (2019)
- [Conditional Random Field Autoencoders for Unsupervised Structured Prediction](https://mlsanity.com/paper/1411.1147/) (2014)
- [Approximation-Aware Dependency Parsing by Belief Propagation](https://mlsanity.com/paper/1508.02375/) (2015)
- [Neuralizing Efficient Higher-order Belief Propagation](https://mlsanity.com/paper/2010.09283/) (2020)
- [Sparse Fuzzy Attention for Structured Sentiment Analysis](https://mlsanity.com/paper/2109.06719/) (2021)

## Built on by (most cited first, 1 of 1)

- [Dynamic Programming in Rank Space: Scaling Structured Inference with Low-Rank HMMs and PCFGs](https://mlsanity.com/paper/2205.00484/) (2022)

## Abstract (arXiv)

Second-order semantic parsing with end-to-end mean-field inference has been shown good performance. In this work we aim to improve this method by modeling label correlations between adjacent arcs. However, direct modeling leads to memory explosion because second-order score tensors have sizes of O(n3L2) (n is the sentence length and L is the number of labels), which is not affordable. To tackle this computational challenge, we leverage tensor decomposition techniques, and interestingly, we show that the large second-order score tensors have no need to be materialized during mean-field inference, thereby reducing the computational complexity from cubic to quadratic. We conduct experiments on SemEval 2015 Task 18 English datasets, showing the effectiveness of modeling label correlations. Our code is publicly available at https://github.com/sustcsonglin/mean-field-dep-parsing.