# Structured Tree Alignment for Evaluation of (Speech) Constituency Parsing

arXiv: https://arxiv.org/abs/2402.13433 · HTML: https://mlsanity.com/paper/2402.13433/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 42 references that are in the map.
- First posted on arXiv 21 Feb 2024.

## Builds on (most cited first, 13 of 13)

- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Constituency Parsing with a Self-Attentive Encoder](https://mlsanity.com/paper/1805.01052/) (2018)
- [Multilingual Constituency Parsing with Self-Attention and Pre-Training](https://mlsanity.com/paper/1812.11760/) (2018)
- [Recurrent Neural Network Grammars](https://mlsanity.com/paper/1602.07776/) (2016)
- [Compound Probabilistic Context-Free Grammars for Grammar Induction](https://mlsanity.com/paper/1906.10225/) (2019)
- [A Minimal Span-Based Neural Constituency Parser](https://mlsanity.com/paper/1705.03919/) (2017)
- [Bidimensional Leaderboards: Generate and Evaluate Language Hand in Hand](https://mlsanity.com/paper/2112.04139/) (2021)
- [Improving Disfluency Detection by Self-Training a Self-Attentive Model](https://mlsanity.com/paper/2004.05323/) (2020)
- [Span-Based Constituency Parsing with a Structure-Label System and Provably Optimal Dynamic Oracles](https://mlsanity.com/paper/1612.06475/) (2016)
- [On the Role of Supervision in Unsupervised Constituency Parsing](https://mlsanity.com/paper/2010.02423/) (2020)
- [Neural CRF Parsing](https://mlsanity.com/paper/1507.03641/) (2015)
- [Assessing the Use of Prosody in Constituency Parsing of Imperfect Transcripts](https://mlsanity.com/paper/2106.07794/) (2021)
- [Cascading and Direct Approaches to Unsupervised Constituency Parsing on Spoken Sentences](https://mlsanity.com/paper/2303.08809/) (2023)

## Built on by (most cited first, 1 of 1)

- [Investigating the (De)Composition Capabilities of Large Language Models in Natural-to-Formal Language Conversion](https://mlsanity.com/paper/2501.14649/) (2025)

## Abstract (arXiv)

We present the structured average intersection-over-union ratio (STRUCT-IOU), a similarity metric between constituency parse trees motivated by the problem of evaluating speech parsers. STRUCT-IOU enables comparison between a constituency parse tree (over automatically recognized spoken word boundaries) with the ground-truth parse (over written words). To compute the metric, we project the ground-truth parse tree to the speech domain by forced alignment, align the projected ground-truth constituents with the predicted ones under certain structured constraints, and calculate the average IOU score across all aligned constituent pairs. STRUCT-IOU takes word boundaries into account and overcomes the challenge that the predicted words and ground truth may not have perfect one-to-one correspondence. Extending to the evaluation of text constituency parsing, we demonstrate that STRUCT-IOU can address token-mismatch issues, and shows higher tolerance to syntactically plausible parses than PARSEVAL (Black et al., 1991).