mlsanityData to 7 Oct 2026

Investigating Post-pretraining Representation Alignment for Cross-Lingual Question Answering

arXiv 2109.12028 ↗First posted 24 Sep 2021cs.CLcited by 3

Not on the map. Why?

Abstract (arXiv)

Human knowledge is collectively encoded in the roughly 6500 languages spoken around the world, but it is not distributed equally across languages. Hence, for information-seeking question answering (QA) systems to adequately serve speakers of all languages, they need to operate cross-lingually. In this work we investigate the capabilities of multilingually pre-trained language models on cross-lingual QA. We find that explicitly aligning the representations across languages with a post-hoc fine-tuning step generally leads to improved performance. We additionally investigate the effect of data size as well as the language choice in this fine-tuning step, also releasing a dataset for evaluating cross-lingual QA systems. Code and dataset are publicly available here: https://github.com/ffaisal93/aligned_qa

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 27 of its 38 references that are in the map. First posted on arXiv 24 Sep 2021.

Builds on

27 of 38 references.

References · 27

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  4. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  5. 2019Cross-lingual Language Model Pretrainingcited by 1,088
  6. 2019On the Cross-lingual Transferability of Monolingual Representationscited by 575
  7. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 486
  8. 2019MLQA: Evaluating Cross-lingual Extractive Question Answeringcited by 339
  9. 2019WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipediacited by 221
  10. 2021Word Alignment by Fine-tuning Embeddings on Parallel Corporacited by 186
  11. 2018Neural Approaches to Conversational AIcited by 164
  12. 2020MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answeringcited by 151
15 more
  1. 2019CCAligned: A Massive Collection of Cross-Lingual Web-Document Pairscited by 136
  2. 2020XOR QA: Cross-lingual Open-Retrieval Question Answeringcited by 121
  3. 2020X-FACTR: Multilingual Factual Knowledge Retrieval from Pretrained Language Modelscited by 117
  4. 2020Multilingual Alignment of Contextual Word Representationscited by 113
  5. 2021QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehensioncited by 91
  6. 2020Inducing Language-Agnostic Multilingual Representationscited by 48
  7. 2020On the Limitations of Cross-lingual Encoders as Exposed by Reference-Free Machine Translation Evaluationcited by 36
  8. 2019A BERT Baseline for the Natural Questionscited by 33
  9. 2019Evaluating the Cross-Lingual Effectiveness of Massively Multilingual Neural Machine Translationcited by 27
  10. 2021NoiseQA: Challenge Set Evaluation for User-Centric Question Answeringcited by 25
  11. 2020Cross-lingual Alignment Methods for Multilingual BERT: A Comparative Studycited by 24
  12. 2020On Negative Interference in Multilingual Models: Findings and A Meta-Learning Treatmentcited by 21
  13. 2021Improving Zero-Shot Cross-Lingual Transfer Learning via Robust Trainingcited by 21
  14. 2019Should All Cross-Lingual Embeddings Speak English?cited by 14
  15. 2021Towards More Equitable Question Answering Systems: How Much More Data Do You Need?cited by 8

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 0; 2024: 2.

3 papers.

  1. 2024On the Scaling Laws of Geographical Representation in Language Modelscited by 14
  2. 2022The Impact of Cross-Lingual Adjustment of Contextual Word Representations on Zero-Shot Transfercited by 5
  3. 2024Birdie: Advancing State Space Models with Reward-Driven Objectives and Curriculacited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025CALM: Unleashing the Cross-Lingual Self-Aligning Ability of Language Model Question Answering
  2. 2023On the Calibration of Multilingual Question Answering LLMs
  3. 2024The Power of Question Translation Training in Multilingual Reasoning: Broadened Scope and Deepened Insights
  4. 2023Bridging the Language Gap: Knowledge Injected Multilingual Question Answering
  5. 2023Cross-lingual QA: A Key to Unlocking In-context Cross-lingual Performance
  6. 2026XHotpotQA: A Benchmark for Cross-Lingual Knowledge Composition in Multi-Hop Question Answering
  7. 2024Question Translation Training for Better Multilingual Reasoning
  8. 2026MSQA: A Natively Sourced Multilingual and Multicultural SimpleQA Benchmark