mlsanityData to 14 Oct 2026

Compositional Generalization in Multilingual Semantic Parsing over Wikidata

arXiv 2108.03509 ↗First posted 7 Aug 2021cs.CLcited by 9

Not on the map. Why?

Abstract (arXiv)

Semantic parsing (SP) allows humans to leverage vast knowledge resources through natural interaction. However, parsers are mostly designed for and evaluated on English resources, such as CFQ (Keysers et al., 2020), the current standard benchmark based on English data generated from grammar rules and oriented towards Freebase, an outdated knowledge base. We propose a method for creating a multilingual, parallel dataset of question-query pairs, grounded in Wikidata. We introduce such a dataset, which we call Multilingual Compositional Wikidata Questions (MCWQ), and use it to analyze the compositional generalization of semantic parsers in Hebrew, Kannada, Chinese and English. While within-language generalization is comparable across languages, experiments on zero-shot cross-lingual transfer demonstrate that cross-lingual compositional generalization fails, even with state-of-the-art pretrained multilingual encoders. Furthermore, our methodology, dataset and results will facilitate future research on SP in more realistic and diverse settings than has been possible with existing resources.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 48 of its 90 references that are in the map. First posted on arXiv 7 Aug 2021.

Builds on

48 of 90 references.

References · 48

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  5. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  6. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  7. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  8. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 903
  9. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 798
  10. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 742
  11. 2020Knowledge Graphscited by 621
  12. 2019On the Cross-lingual Transferability of Monolingual Representationscited by 578
36 more
  1. 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 445
  2. 2018The Web as a Knowledge-base for Answering Complex Questionscited by 410
  3. 2019Measuring Compositional Generalization: A Comprehensive Method on Realistic Datacited by 284
  4. 2020COGS: A Compositional Generalization Challenge Based on Semantic Interpretationcited by 263
  5. 2020XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generationcited by 222
  6. 2020Are All Languages Created Equal in Multilingual BERT?cited by 207
  7. 2019The Evolved Transformercited by 180
  8. 2020Beyond I.I.D.: Three Levels of Generalization for Question Answering on Knowledge Basescited by 171
  9. 2021XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluationcited by 168
  10. 2019Choosing Transfer Languages for Cross-Lingual Learningcited by 144
  11. 2015Large-scale Simple Question Answering with Memory Networkscited by 127
  12. 2020A Benchmark for Systematic Generalization in Grounded Language Understandingcited by 108
  13. 2021Case-based Reasoning for Natural Language Queries over Knowledge Basescited by 107
  14. 2020Translation Artifacts in Cross-lingual Transfer Learningcited by 96
  15. 2018Tensor2Tensor for Neural Machine Translationcited by 85
  16. 2020KQA Pro: A Dataset with Explicit Compositional Programs for Complex Question Answering over Knowledge Basecited by 78
  17. 2020Span-based Semantic Parsing for Compositional Generalizationcited by 77
  18. 2018Complex Sequential Question Answering: Towards Learning to Converse Over Linked Question Answer Pairs with a Knowledge Graphcited by 67
  19. 2020FQuAD: French Question Answering Datasetcited by 58
  20. 2018DRCD: a Chinese Machine Reading Comprehension Datasetcited by 57
  21. 2021Unlocking Compositional Generalization in Pre-trained Models Using Intermediate Representationscited by 55
  22. 2020Improving Compositional Generalization in Semantic Parsingcited by 42
  23. 2019Multi-Task Learning for Conversational Question Answering over a Large-Scale Knowledge Basecited by 33
  24. 2020Hierarchical Poset Decoding for Compositional Generalization in Languagecited by 26
  25. 2019Compositional Generalization in Image Captioningcited by 23
  26. 2021Database Reasoning Over Textcited by 20
  27. 2020Revisiting Iterative Back-Translation from the Perspective of Compositional Generalizationcited by 19
  28. 2021Zero-Shot Cross-lingual Semantic Parsingcited by 19
  29. 2020*-CFQ: Analyzing the Scalability of Machine Learning on a Compositional Taskcited by 16
  30. 2020RuBQ: A Russian Dataset for Question Answering over Wikidatacited by 14
  31. 2021CHOLAN: A Modular Approach for Neural Entity Linking on Wikipedia and Wikidatacited by 14
  32. 2016Cross-Lingual Syntactic Transfer with Limited Resourcescited by 8
  33. 2017Neural Semantic Parsing over Multiple Knowledge-basescited by 8
  34. 2017Cross-Lingual Sentiment Analysis Without (Good) Translationcited by 7
  35. 2018Weakly-supervised Neural Semantic Parsing with a Generative Rankercited by 5
  36. 2019Rewarding Coreference Resolvers for Being Consistent with World Knowledgecited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 4; 2024: 0; 2025: 2; 2026: 2.

9 papers.

  1. 2022XRICL: Cross-lingual Retrieval-Augmented In-Context Learning for Cross-lingual Text-to-SQL Semantic Parsingcited by 33
  2. 2023Pay More Attention to Relation Exploration for Knowledge Base Question Answeringcited by 7
  3. 2023On Evaluating Multilingual Compositional Generalization with Translated Datasetscited by 6
  4. 2025SPARQL Query Generation with LLMs: Measuring the Impact of Training Data Memorization and Knowledge Injectioncited by 6
  5. 2023XSemPLR: Cross-Lingual Semantic Parsing in Multiple Natural Languages and Meaning Representationscited by 5
  6. 2025Text2Cypher Across Languages: Evaluating and Finetuning LLMscited by 3
  7. 2023GraphextQA: A Benchmark for Evaluating Graph-Enhanced Large Language Modelscited by 2
  8. 2026Incremental Multilingual Text2Cypher with Adapter Combinationcited by 0
  9. 2026Known Intents, New Combinations: Clause-Factorized Decoding for Compositional Multi-Intent Detectioncited by 0