# Non-Autoregressive Semantic Parsing for Compositional Task-Oriented Dialog

arXiv: https://arxiv.org/abs/2104.04923 · HTML: https://mlsanity.com/paper/2104.04923/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 35 references that are in the map.
- First posted on arXiv 11 Apr 2021.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [Sequence-Level Knowledge Distillation](https://mlsanity.com/paper/1606.07947/) (2016)
- [FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search](https://mlsanity.com/paper/1812.03443/) (2018)
- [Regularizing Neural Networks by Penalizing Confident Output Distributions](https://mlsanity.com/paper/1701.06548/) (2017)
- [Non-Autoregressive Neural Machine Translation](https://mlsanity.com/paper/1711.02281/) (2017)
- [Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces](https://mlsanity.com/paper/1805.10190/) (2018)
- [Pay Less Attention with Lightweight and Dynamic Convolutions](https://mlsanity.com/paper/1901.10430/) (2019)
- [Levenshtein Transformer](https://mlsanity.com/paper/1905.11006/) (2019)
- [Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinement](https://mlsanity.com/paper/1802.06901/) (2018)
- [Understanding Knowledge Distillation in Non-autoregressive Machine Translation](https://mlsanity.com/paper/1911.02727/) (2019)
- [Semantic Parsing for Task Oriented Dialog using Hierarchical Representations](https://mlsanity.com/paper/1810.07942/) (2018)
- [Non-Autoregressive Machine Translation with Latent Alignments](https://mlsanity.com/paper/2004.07437/) (2020)
- [FlowSeq: Non-Autoregressive Conditional Sequence Generation with Generative Flow](https://mlsanity.com/paper/1909.02480/) (2019)
- [Joint Slot Filling and Intent Detection via Capsule Neural Networks](https://mlsanity.com/paper/1812.09471/) (2018)
- [Aligned Cross Entropy for Non-Autoregressive Machine Translation](https://mlsanity.com/paper/2004.01655/) (2020)
- [Don't Parse, Generate! A Sequence to Sequence Architecture for Task-Oriented Semantic Parsing](https://mlsanity.com/paper/2001.11458/) (2020)
- [Global-Locally Self-Attentive Dialogue State Tracker](https://mlsanity.com/paper/1805.09655/) (2018)
- [Conversational Semantic Parsing](https://mlsanity.com/paper/2009.13655/) (2020)
- [Semi-Autoregressive Training Improves Mask-Predict Decoding](https://mlsanity.com/paper/2001.08785/) (2020)
- [Improving Semantic Parsing for Task Oriented Dialog](https://mlsanity.com/paper/1902.06000/) (2019)
- [Non-Autoregressive Machine Translation with Disentangled Context Transformer](https://mlsanity.com/paper/2001.05136/) (2020)
- [On Optimal Transformer Depth for Low-Resource Language Translation](https://mlsanity.com/paper/2004.04418/) (2020)
- [Auto-Sizing the Transformer Network: Improving Speed, Efficiency, and Performance for Low-Resource Machine Translation](https://mlsanity.com/paper/1910.06717/) (2019)

## Built on by (most cited first, 17 of 17)

- [A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyond](https://mlsanity.com/paper/2204.09269/) (2022)
- [Span Pointer Networks for Non-Autoregressive Task-Oriented Semantic Parsing](https://mlsanity.com/paper/2104.07275/) (2021)
- [Latency-Aware Neural Architecture Search with Multi-Objective Bayesian Optimization](https://mlsanity.com/paper/2106.11890/) (2021)
- [Chaining Simultaneous Thoughts for Numerical Reasoning](https://mlsanity.com/paper/2211.16482/) (2022)
- [Semantic Parsing in Task-Oriented Dialog with Recursive Insertion-based Encoder](https://mlsanity.com/paper/2109.04500/) (2021)
- [Deliberation Model for On-Device Spoken Language Understanding](https://mlsanity.com/paper/2204.01893/) (2022)
- [Retrieve-and-Fill for Scenario-based Task-Oriented Semantic Parsing](https://mlsanity.com/paper/2202.00901/) (2022)
- [Any2Graph: Deep End-To-End Supervised Graph Prediction With An Optimal Transport Loss](https://mlsanity.com/paper/2402.12269/) (2024)
- [Reducing Model Jitter: Stable Re-training of Semantic Parsers in Production Environments](https://mlsanity.com/paper/2204.04735/) (2022)
- [X2Parser: Cross-Lingual and Cross-Domain Framework for Task-Oriented Compositional Semantic Parsing](https://mlsanity.com/paper/2106.03777/) (2021)
- [Improving Top-K Decoding for Non-Autoregressive Semantic Parsing via Intent Conditioning](https://mlsanity.com/paper/2204.06748/) (2022)
- [TreePiece: Faster Semantic Parsing via Tree Tokenization](https://mlsanity.com/paper/2303.17161/) (2023)
- [Transfer Learning for Multi-lingual Tasks -- a Survey](https://mlsanity.com/paper/2110.02052/) (2021)
- [Text generation for dataset augmentation in security classification tasks](https://mlsanity.com/paper/2310.14429/) (2023)
- [Assessing Data Efficiency in Task-Oriented Semantic Parsing](https://mlsanity.com/paper/2107.04736/) (2021)
- [Shift-Reduce Task-Oriented Semantic Parsing with Stack-Transformers](https://mlsanity.com/paper/2210.11984/) (2022)
- [PRoDeliberation: Parallel Robust Deliberation for End-to-End Spoken Language Understanding](https://mlsanity.com/paper/2406.07823/) (2024)

## Abstract (arXiv)

Semantic parsing using sequence-to-sequence models allows parsing of deeper representations compared to traditional word tagging based models. In spite of these advantages, widespread adoption of these models for real-time conversational use cases has been stymied by higher compute requirements and thus higher latency. In this work, we propose a non-autoregressive approach to predict semantic parse trees with an efficient seq2seq model architecture. By combining non-autoregressive prediction with convolutional neural networks, we achieve significant latency gains and parameter size reduction compared to traditional RNN models. Our novel architecture achieves up to an 81% reduction in latency on TOP dataset and retains competitive performance to non-pretrained models on three different semantic parsing datasets. Our code is available at https://github.com/facebookresearch/pytext