mlsanityData to 7 Oct 2026

Translate & Fill: Improving Zero-Shot Multilingual Semantic Parsing with Synthetic Data

arXiv 2109.04319 ↗First posted 9 Sep 2021cs.CL cs.AI cs.LGcited by 17

On the map: Synthetic data and self-improvement for LLMs → LLM synthetic data generation

Abstract (arXiv)

While multilingual pretrained language models (LMs) fine-tuned on a single language have shown substantial cross-lingual task transfer capabilities, there is still a wide performance gap in semantic parsing tasks when target language supervision is available. In this paper, we propose a novel Translate-and-Fill (TaF) method to produce silver training data for a multilingual semantic parser. This method simplifies the popular Translate-Align-Project (TAP) pipeline and consists of a sequence-to-sequence filler model that constructs a full parse conditioned on an utterance and a view of the same parse. Our filler is trained on English data only but can accurately complete instances in other languages (i.e., translations of the English training utterances), in a zero-shot fashion. Experimental results on three multilingual semantic parsing datasets show that data augmentation with TaF reaches accuracies competitive with similar systems which rely on traditional alignment techniques.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Placed in LLM synthetic data generation, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 38 of its 62 references that are in the map. First posted on arXiv 9 Sep 2021.

Builds on

38 of 62 references.

References · 38

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  4. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  5. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  6. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  7. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  8. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
  9. 2019Cross-lingual Language Model Pretrainingcited by 1,088
  10. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 738
  11. 2021CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representationcited by 229
  12. 2021Word Alignment by Fine-tuning Embeddings on Parallel Corporacited by 186
26 more
  1. 2021Structured Prediction as Translation between Augmented Natural Languagescited by 183
  2. 2020MTOP: A Comprehensive Multilingual Task-Oriented Semantic Parsing Benchmarkcited by 156
  3. 2020SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddingscited by 155
  4. 2020WikiLingua: A New Benchmark Dataset for Cross-Lingual Abstractive Summarizationcited by 154
  5. 2018Cross-Lingual Transfer Learning for Multilingual Task Oriented Dialogcited by 124
  6. 2018Coarse-to-Fine Decoding for Neural Semantic Parsingcited by 110
  7. 2019A Stack-Propagation Framework with Token-Level Intent Detection for Spoken Language Understandingcited by 106
  8. 2018Semantic Parsing for Task Oriented Dialog using Hierarchical Representationscited by 101
  9. 2020End-to-End Slot Alignment and Recognition for Cross-Lingual NLUcited by 82
  10. 2018Learning Neural Templates for Text Generationcited by 61
  11. 2020Don't Parse, Generate! A Sequence to Sequence Architecture for Task-Oriented Semantic Parsingcited by 59
  12. 2020Cross-Lingual Semantic Role Labeling with High-Quality Translated Training Corpuscited by 49
  13. 2018Neural Cross-Lingual Named Entity Recognition with Minimal Resourcescited by 44
  14. 2017Weakly Supervised Cross-Lingual Named Entity Recognition via Effective Annotation and Representation Projectioncited by 36
  15. 2019Entity Projection via Machine Translation for Cross-Lingual NERcited by 32
  16. 2020Accurate Word Alignment Induction from Neural Machine Translationcited by 32
  17. 2020End-to-End Neural Word Alignment Outperforms GIZA++cited by 28
  18. 2019Evaluating the Cross-Lingual Effectiveness of Massively Multilingual Neural Machine Translationcited by 27
  19. 2018Zero-Shot Cross-lingual Classification Using Multilingual Neural Machine Translationcited by 19
  20. 2020X-SRL: A Parallel Cross-Lingual Semantic Role Labeling Datasetcited by 15
  21. 2019Cross-lingual Dependency Parsing with Unlabeled Auxiliary Languagescited by 13
  22. 2019Cross-Lingual Dependency Parsing Using Code-Mixed TreeBankcited by 10
  23. 2021Distilling Large Language Models into Tiny and Effective Students using pQRNNcited by 10
  24. 2018Distant Supervision from Disparate Sources for Low-Resource Part-of-Speech Taggingcited by 8
  25. 2020Weakly Supervised POS Taggers Perform Poorly on Truly Low-Resource Languagescited by 7
  26. 2017Transfer Learning for Neural Semantic Parsingcited by 5

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 9; 2023: 5; 2024: 2.

17 papers.

Sort:

By area:

  1. 2022MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languagescited by 142
  2. 2022Frustratingly Easy Label Projection for Cross-lingual Transfercited by 33
  3. 2023X-RiSAWOZ: High-Quality End-to-End Multilingual Dialogue Datasets and Few-shot Agentscited by 12
  4. 2022Meta-Learning a Cross-lingual Manifold for Semantic Parsingcited by 11
  5. 2023Contextual Label Projection for Cross-Lingual Structured Predictioncited by 9
  6. 2022Retrieve-and-Fill for Scenario-based Task-Oriented Semantic Parsingcited by 7
  7. 2022Unveiling the Black Box of PLMs with Semantic Anchors: Towards Interpretable Neural Semantic Parsingcited by 5
  8. 2022CST5: Data Augmentation for Code-Switched Semantic Parsingcited by 5
  9. 2022Z-BERT-A: a zero-shot Pipeline for Unknown Intent detectioncited by 4
  10. 2021Call Larisa Ivanovna: Code-Switching Fools Multilingual NLU Modelscited by 2
  11. 2022Bootstrapping Multilingual Semantic Parsers using Large Language Modelscited by 2
  12. 2023Optimal Transport Posterior Alignment for Cross-lingual Semantic Parsingcited by 2
5 more
  1. 2022DAMP: Doubly Aligned Multilingual Parser for Task-Oriented Dialoguecited by 1
  2. 2023Evaluating Inter-Bilingual Semantic Parsing for Indian Languagescited by 1
  3. 2024Task-Oriented Dialog Systems for the Senegalese Wolof Languagecited by 1
  4. 2023Semantic Parsing in Limited Resource Conditionscited by 0
  5. 2024Cross-lingual Back-Parsing: Utterance Synthesis from Meaning Representation for Zero-Resource Semantic Parsingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Translate First Reorder Later: Leveraging Monotonicity in Semantic Parsing
  2. 2021Hierarchical Neural Data Synthesis for Semantic Parsing
  3. 2024Pruning Multilingual Large Language Models for Multilingual Inference
  4. 2022XRICL: Cross-lingual Retrieval-Augmented In-Context Learning for Cross-lingual Text-to-SQL Semantic Parsing
  5. 2025Benchmarking Cross-Lingual Semantic Alignment in Multilingual Embeddings
  6. 2026IFMTBench: A Comprehensive Benchmark for Multilingual Translation Instruction Following
  7. 2025Multilingual Language Model Pretraining using Machine-translated Data
  8. 2020Multilingual BERT Post-Pretraining Alignment