# One Semantic Parser to Parse Them All: Sequence to Sequence Multi-Task Learning on Semantic Parsing Datasets

arXiv: https://arxiv.org/abs/2106.04476 · HTML: https://mlsanity.com/paper/2106.04476/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 67 references that are in the map.
- First posted on arXiv 8 Jun 2021.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Pointer Networks](https://mlsanity.com/paper/1506.03134/) (2015)
- [An Overview of Multi-Task Learning in Deep Neural Networks](https://mlsanity.com/paper/1706.05098/) (2017)
- [Green AI](https://mlsanity.com/paper/1907.10597/) (2019)
- [Exploring Generalization in Deep Learning](https://mlsanity.com/paper/1706.08947/) (2017)
- [Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation](https://mlsanity.com/paper/1611.04558/) (2016)
- [Measuring Compositional Generalization: A Comprehensive Method on Realistic Data](https://mlsanity.com/paper/1912.09713/) (2019)
- [Transfer Learning for Low-Resource Neural Machine Translation](https://mlsanity.com/paper/1604.02201/) (2016)
- [Improving Text-to-SQL Evaluation Methodology](https://mlsanity.com/paper/1806.09029/) (2018)
- [Data Recombination for Neural Semantic Parsing](https://mlsanity.com/paper/1606.03622/) (2016)
- [Semantic Parsing for Task Oriented Dialog using Hierarchical Representations](https://mlsanity.com/paper/1810.07942/) (2018)
- [Multi-task Sequence to Sequence Learning](https://mlsanity.com/paper/1511.06114/) (2015)
- [Pre-training on high-resource speech recognition improves low-resource speech-to-text translation](https://mlsanity.com/paper/1809.01431/) (2018)
- [Identifying beneficial task relations for multi-task learning in deep neural networks](https://mlsanity.com/paper/1702.08303/) (2017)
- [Don't Parse, Generate! A Sequence to Sequence Architecture for Task-Oriented Semantic Parsing](https://mlsanity.com/paper/2001.11458/) (2020)
- [Neural AMR: Sequence-to-Sequence Models for Parsing and Generation](https://mlsanity.com/paper/1704.08381/) (2017)
- [A Hierarchical Multi-task Approach for Learning Embeddings from Semantic Tasks](https://mlsanity.com/paper/1811.06031/) (2018)
- [AMR Parsing via Graph-Sequence Iterative Inference](https://mlsanity.com/paper/2004.05572/) (2020)
- [The Parallel Meaning Bank: Towards a Multilingual Corpus of Translations Annotated with Compositional Meaning Representations](https://mlsanity.com/paper/1702.03964/) (2017)
- [Conversational Semantic Parsing](https://mlsanity.com/paper/2009.13655/) (2020)
- [An Incremental Parser for Abstract Meaning Representation](https://mlsanity.com/paper/1608.06111/) (2016)
- [Improving a Neural Semantic Parser by Counterfactual Learning from Human Bandit Feedback](https://mlsanity.com/paper/1805.01252/) (2018)
- [Broad-Coverage Semantic Parsing as Transduction](https://mlsanity.com/paper/1909.02607/) (2019)
- [Lambda Dependency-Based Compositional Semantics](https://mlsanity.com/paper/1309.4408/) (2013)
- [When is multitask learning effective? Semantic sequence prediction under varying data conditions](https://mlsanity.com/paper/1612.02251/) (2016)

## Built on by (most cited first, 3 of 3)

- [State-of-the-art generalisation research in NLP: A taxonomy and review](https://mlsanity.com/paper/2210.03050/) (2022)
- [GraphQ IR: Unifying the Semantic Parsing of Graph Query Languages with One Intermediate Representation](https://mlsanity.com/paper/2205.12078/) (2022)
- [ATP: AMRize Then Parse! Enhancing AMR Parsing with PseudoAMRs](https://mlsanity.com/paper/2204.08875/) (2022)

## Abstract (arXiv)

Semantic parsers map natural language utterances to meaning representations. The lack of a single standard for meaning representations led to the creation of a plethora of semantic parsing datasets. To unify different datasets and train a single model for them, we investigate the use of Multi-Task Learning (MTL) architectures. We experiment with five datasets (Geoquery, NLMaps, TOP, Overnight, AMR). We find that an MTL architecture that shares the entire network across datasets yields competitive or better parsing accuracies than the single-task baselines, while reducing the total number of parameters by 68%. We further provide evidence that MTL has also better compositional generalization than single-task models. We also present a comparison of task sampling methods and propose a competitive alternative to widespread proportional sampling strategies.