# Parsing Speech: A Neural Approach to Integrating Lexical and Acoustic-Prosodic Information

arXiv: https://arxiv.org/abs/1704.07287 · HTML: https://mlsanity.com/paper/1704.07287/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Apr 2017.

## Built on by (most cited first, 2 of 2)

- [Fillers in Spoken Language Understanding: Computational and Psycholinguistic Perspectives](https://mlsanity.com/paper/2301.10761/) (2023)
- [Cross-stitched Multi-modal Encoders](https://mlsanity.com/paper/2204.09227/) (2022)

## Abstract (arXiv)

In conversational speech, the acoustic signal provides cues that help listeners disambiguate difficult parses. For automatically parsing spoken utterances, we introduce a model that integrates transcribed text and acoustic-prosodic features using a convolutional neural network over energy and pitch trajectories coupled with an attention-based recurrent neural network that accepts text and prosodic features. We find that different types of acoustic-prosodic features are individually helpful, and together give statistically significant improvements in parse and disfluency detection F1 scores over a strong text-only baseline. For this study with known sentence boundaries, error analyses show that the main benefit of acoustic-prosodic features is in sentences with disfluencies, attachment decisions are most improved, and transcription errors obscure gains from prosody.