# Structural Transfer Learning in NL-to-Bash Semantic Parsers

arXiv: https://arxiv.org/abs/2307.16795 · HTML: https://mlsanity.com/paper/2307.16795/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 10 references that are in the map.
- First posted on arXiv 31 Jul 2023.

## Builds on (most cited first, 8 of 8)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task](https://mlsanity.com/paper/1809.08887/) (2018)
- [Effective Approaches to Attention-based Neural Machine Translation](https://mlsanity.com/paper/1508.04025/) (2015)
- [GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing](https://mlsanity.com/paper/2009.13845/) (2020)
- [Learning to Mine Aligned Code and Natural Language Pairs from Stack Overflow](https://mlsanity.com/paper/1805.08949/) (2018)
- [Learning Contextual Representations for Semantic Parsing with Generation-Augmented Pre-Training](https://mlsanity.com/paper/2012.10309/) (2020)
- [Incorporating External Knowledge through Pre-training for Natural Language to Code Generation](https://mlsanity.com/paper/2004.09015/) (2020)
- [NeurIPS 2020 NLC2CMD Competition: Translating Natural Language to Bash Commands](https://mlsanity.com/paper/2103.02523/) (2021)

## Abstract (arXiv)

Large-scale pre-training has made progress in many fields of natural language processing, though little is understood about the design of pre-training datasets. We propose a methodology for obtaining a quantitative understanding of structural overlap between machine translation tasks. We apply our methodology to the natural language to Bash semantic parsing task (NLBash) and show that it is largely reducible to lexical alignment. We also find that there is strong structural overlap between NLBash and natural language to SQL. Additionally, we perform a study varying compute expended during pre-training on the English to German machine translation task and find that more compute expended during pre-training does not always correspond semantic representations with stronger transfer to NLBash.