# Combining Improvements for Exploiting Dependency Trees in Neural Semantic Parsing

arXiv: https://arxiv.org/abs/2112.13179 · HTML: https://mlsanity.com/paper/2112.13179/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 30 references that are in the map.
- First posted on arXiv 25 Dec 2021.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Deep Biaffine Attention for Neural Dependency Parsing](https://mlsanity.com/paper/1611.01734/) (2016)
- [Language to Logical Form with Neural Attention](https://mlsanity.com/paper/1601.01280/) (2016)
- [Data Recombination for Neural Semantic Parsing](https://mlsanity.com/paper/1606.03622/) (2016)
- [Coarse-to-Fine Decoding for Neural Semantic Parsing](https://mlsanity.com/paper/1805.04793/) (2018)
- [TRANX: A Transition-based Neural Abstract Syntax Parser for Semantic Parsing and Code Generation](https://mlsanity.com/paper/1810.02720/) (2018)
- [Tree Transformer: Integrating Tree Structures into Self-Attention](https://mlsanity.com/paper/1909.06639/) (2019)
- [TreeGen: A Tree-Based Transformer Architecture for Code Generation](https://mlsanity.com/paper/1911.09983/) (2019)
- [Simpler but More Accurate Semantic Dependency Parsing](https://mlsanity.com/paper/1807.01396/) (2018)
- [A Grammar-Based Structural CNN Decoder for Code Generation](https://mlsanity.com/paper/1811.06837/) (2018)
- [Enhancing Machine Translation with Dependency-Aware Self-Attention](https://mlsanity.com/paper/1909.03149/) (2019)
- [Universal Semantic Parsing](https://mlsanity.com/paper/1702.03196/) (2017)
- [Sequence-to-Action: End-to-End Semantic Graph Generation for Semantic Parsing](https://mlsanity.com/paper/1809.00773/) (2018)
- [Syntax-Enhanced Neural Machine Translation with Syntax-Aware Word Representations](https://mlsanity.com/paper/1905.02878/) (2019)
- [Generating Logical Forms from Graph Representations of Text and Entities](https://mlsanity.com/paper/1905.08407/) (2019)
- [Exploiting Rich Syntactic Information for Semantic Parsing with Graph-to-Sequence Model](https://mlsanity.com/paper/1808.07624/) (2018)
- [Type-Driven Incremental Semantic Parsing with Polymorphism](https://mlsanity.com/paper/1411.5379/) (2014)

## Abstract (arXiv)

The dependency tree of a natural language sentence can capture the interactions between semantics and words. However, it is unclear whether those methods which exploit such dependency information for semantic parsing can be combined to achieve further improvement and the relationship of those methods when they combine. In this paper, we examine three methods to incorporate such dependency information in a Transformer based semantic parser and empirically study their combinations. We first replace standard self-attention heads in the encoder with parent-scaled self-attention (PASCAL) heads, i.e., the ones that can attend to the dependency parent of each token. Then we concatenate syntax-aware word representations (SAWRs), i.e., the intermediate hidden representations of a neural dependency parser, with ordinary word embedding to enhance the encoder. Later, we insert the constituent attention (CA) module to the encoder, which adds an extra constraint to attention heads that can better capture the inherent dependency structure of input sentences. Transductive ensemble learning (TEL) is used for model aggregation, and an ablation study is conducted to show the contribution of each method. Our experiments show that CA is complementary to PASCAL or SAWRs, and PASCAL + CA provides state-of-the-art performance among neural approaches on ATIS, GEO, and JOBS.