# Shift-Reduce Task-Oriented Semantic Parsing with Stack-Transformers

arXiv: https://arxiv.org/abs/2210.11984 · HTML: https://mlsanity.com/paper/2210.11984/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 33 of its 50 references that are in the map.
- First posted on arXiv 21 Oct 2022.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [Convolutional Sequence to Sequence Learning](https://mlsanity.com/paper/1705.03122/) (2017)
- [Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Filling](https://mlsanity.com/paper/1609.01454/) (2016)
- [Sequence-to-Sequence Learning as Beam-Search Optimization](https://mlsanity.com/paper/1606.02960/) (2016)
- [Recurrent Neural Network Grammars](https://mlsanity.com/paper/1602.07776/) (2016)
- [Semantic Parsing for Task Oriented Dialog using Hierarchical Representations](https://mlsanity.com/paper/1810.07942/) (2018)
- [Low-Resource Domain Adaptation for Compositional Task-Oriented Semantic Parsing](https://mlsanity.com/paper/2010.03546/) (2020)
- [Grammar as a Foreign Language](https://mlsanity.com/paper/1412.7449/) (2014)
- [A Minimal Span-Based Neural Constituency Parser](https://mlsanity.com/paper/1705.03919/) (2017)
- [Don't Parse, Generate! A Sequence to Sequence Architecture for Task-Oriented Semantic Parsing](https://mlsanity.com/paper/2001.11458/) (2020)
- [Transition-Based Dependency Parsing with Stack Long Short-Term Memory](https://mlsanity.com/paper/1505.08075/) (2015)
- [Conversational Semantic Parsing](https://mlsanity.com/paper/2009.13655/) (2020)
- [Transition-based Parsing with Stack-Transformers](https://mlsanity.com/paper/2010.10669/) (2020)
- [In-Order Transition-based Constituent Parsing](https://mlsanity.com/paper/1707.05000/) (2017)
- [Span Pointer Networks for Non-Autoregressive Task-Oriented Semantic Parsing](https://mlsanity.com/paper/2104.07275/) (2021)
- [Non-Autoregressive Semantic Parsing for Compositional Task-Oriented Dialog](https://mlsanity.com/paper/2104.04923/) (2021)
- [AMR Parsing using Stack-LSTMs](https://mlsanity.com/paper/1707.07755/) (2017)
- [Improving Semantic Parsing for Task Oriented Dialog](https://mlsanity.com/paper/1902.06000/) (2019)
- [Don't Parse, Insert: Multilingual Semantic Parsing with Insertion Based Decoding](https://mlsanity.com/paper/2010.03714/) (2020)
- [Faster Shift-Reduce Constituent Parsing with a Non-Binary, Bottom-Up Strategy](https://mlsanity.com/paper/1804.07961/) (2018)

## Abstract (arXiv)

Intelligent voice assistants, such as Apple Siri and Amazon Alexa, are widely used nowadays. These task-oriented dialogue systems require a semantic parsing module in order to process user utterances and understand the action to be performed. This semantic parsing component was initially implemented by rule-based or statistical slot-filling approaches for processing simple queries; however, the appearance of more complex utterances demanded the application of shift-reduce parsers or sequence-to-sequence models. Although shift-reduce approaches were initially considered the most promising option, the emergence of sequence-to-sequence neural systems has propelled them to the forefront as the highest-performing method for this particular task. In this article, we advance the research on shift-reduce semantic parsing for task-oriented dialogue. We implement novel shift-reduce parsers that rely on Stack-Transformers. This framework allows to adequately model transition systems on the Transformer neural architecture, notably boosting shift-reduce parsing performance. Furthermore, our approach goes beyond the conventional top-down algorithm: we incorporate alternative bottom-up and in-order transition systems derived from constituency parsing into the realm of task-oriented parsing. We extensively test our approach on multiple domains from the Facebook TOP benchmark, improving over existing shift-reduce parsers and state-of-the-art sequence-to-sequence models in both high-resource and low-resource settings. We also empirically prove that the in-order algorithm substantially outperforms the commonly-used top-down strategy. Through the creation of innovative transition systems and harnessing the capabilities of a robust neural architecture, our study showcases the superiority of shift-reduce parsers over leading sequence-to-sequence methods on the main benchmark.