mlsanityData to 7 Oct 2026

mPMR: A Multilingual Pre-trained Machine Reader at Scale

arXiv 2305.13645 ↗First posted 23 May 2023cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

We present multilingual Pre-trained Machine Reader (mPMR), a novel method for multilingual machine reading comprehension (MRC)-style pre-training. mPMR aims to guide multilingual pre-trained language models (mPLMs) to perform natural language understanding (NLU) including both sequence classification and span extraction in multiple languages. To achieve cross-lingual generalization when only source-language fine-tuning data is available, existing mPLMs solely transfer NLU capability from a source language to target languages. In contrast, mPMR allows the direct inheritance of multilingual NLU capability from the MRC-style pre-training to downstream tasks. Therefore, mPMR acquires better NLU capability for target languages. mPMR also provides a unified solver for tackling cross-lingual span extraction and sequence classification, thereby enabling the extraction of rationales to explain the sentence-pair classification process.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 24 of its 41 references that are in the map. First posted on arXiv 23 May 2023.

Builds on

24 of 41 references.

References · 24

  1. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  2. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  3. 2019Cross-lingual Language Model Pretrainingcited by 1,088
  4. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 913
  5. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 738
  6. 2019On the Cross-lingual Transferability of Monolingual Representationscited by 575
  7. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 486
  8. 2019MLQA: Evaluating Cross-lingual Extractive Question Answeringcited by 339
  9. 2019PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identificationcited by 262
  10. 2019Emerging Cross-lingual Structure in Pretrained Language Modelscited by 180
  11. 2020Pre-training Tasks for Embedding-based Large-scale Retrievalcited by 153
  12. 2022Lifting the Curse of Multilinguality by Pre-training Modular Transformerscited by 145
12 more
  1. 2020Look at the First Sentence: Position Bias in Question Answeringcited by 83
  2. 2020VECO: Variable and Flexible Cross-lingual Pre-training for Language Understanding and Generationcited by 34
  3. 2021GlobalWoZ: Globalizing MultiWoZ to Develop Multilingual Task-Oriented Dialogue Systemscited by 30
  4. 2020Unsupervised Domain Adaptation of a Pretrained Cross-Lingual Language Modelcited by 24
  5. 2020Unsupervised Cross-lingual Adaptation for Sequence Tagging and Beyondcited by 22
  6. 2021mLUKE: The Power of Entity Representations in Multilingual Pretrained Language Modelscited by 19
  7. 2021XLM-K: Improving Cross-Lingual Language Model Pre-training with Multilingual Knowledgecited by 18
  8. 2021Enhancing Multilingual Language Model with Massive Multilingual Knowledge Triplescited by 13
  9. 2022ConNER: Consistency Training for Cross-lingual Named Entity Recognitioncited by 11
  10. 2021Multilingual AMR Parsing with Noisy Knowledge Distillationcited by 8
  11. 2022PeerDA: Data Augmentation via Modeling Peer Relation for Span Identification Taskscited by 7
  12. 2022Retrofitting Multilingual Sentence Embeddings with Abstract Meaning Representationcited by 4

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 0; 2025: 1.

2 papers.

  1. 2023Reasons to Reject? Aligning Language Models with Judgmentscited by 15
  2. 2025Understanding LLMs' Cross-Lingual Context Retrieval: How Good It Is And Where It Comes Fromcited by 4