mlsanityData to 7 Oct 2026

Transfer Learning for Multi-lingual Tasks -- a Survey

arXiv 2110.02052 ↗First posted 28 Aug 2021cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

These days different platforms such as social media provide their clients from different backgrounds and languages the possibility to connect and exchange information. It is not surprising anymore to see comments from different languages in posts published by international celebrities or data providers. In this era, understanding cross languages content and multilingualism in natural language processing (NLP) are hot topics, and multiple efforts have tried to leverage existing technologies in NLP to tackle this challenging research problem. In this survey, we provide a comprehensive overview of the existing literature with a focus on transfer learning techniques in multilingual tasks. We also identify potential opportunities for further research in this domain.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 63 of its 106 references that are in the map. First posted on arXiv 28 Aug 2021.

Builds on

63 of 106 references.

References · 63

  1. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  2. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  3. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  4. 2020Longformer: The Long-Document Transformercited by 3,362
  5. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  6. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
  7. 2014Convolutional Neural Networks for Sentence Classificationcited by 1,124
  8. 2019Cross-lingual Language Model Pretrainingcited by 1,088
  9. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 913
  10. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 652
  11. 2019How multilingual is Multilingual BERT?cited by 642
  12. 2013Generating Sequences With Recurrent Neural Networkscited by 549
38 more
  1. 2019ERNIE: Enhanced Language Representation with Informative Entitiescited by 525
  2. 2020Pre-trained Models for Natural Language Processing: A Surveycited by 504
  3. 2021ByT5: Towards a token-free future with pre-trained byte-to-byte modelscited by 504
  4. 2018Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfacescited by 376
  5. 2019CamemBERT: a Tasty French Language Modelcited by 368
  6. 2020AraBERT: Transformer-based Model for Arabic Language Understandingcited by 359
  7. 2019Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERTcited by 301
  8. 2019BERT for Joint Intent Classification and Slot Fillingcited by 195
  9. 2019Cross-Lingual Ability of Multilingual BERT: An Empirical Studycited by 182
  10. 2014A Convolutional Neural Network for Modelling Sentencescited by 163
  11. 2020MTOP: A Comprehensive Multilingual Task-Oriented Semantic Parsing Benchmarkcited by 156
  12. 2019FlauBERT: Unsupervised Language Model Pre-training for Frenchcited by 153
  13. 2020PhoBERT: Pre-trained language models for Vietnamesecited by 148
  14. 2017A Survey Of Cross-lingual Word Embedding Modelscited by 142
  15. 2020Beat the AI: Investigating Adversarial Human Annotation for Reading Comprehensioncited by 131
  16. 2018Cross-Lingual Transfer Learning for Multilingual Task Oriented Dialogcited by 124
  17. 2019Multilingual is not enough: BERT for Finnishcited by 118
  18. 2017DuReader: a Chinese Machine Reading Comprehension Dataset from Real-world Applicationscited by 110
  19. 2019Adaptation of Deep Bidirectional Multilingual Transformers for Russian Languagecited by 109
  20. 2019BERTje: A Dutch BERT Modelcited by 109
  21. 201975 Languages, 1 Model: Parsing Universal Dependencies Universallycited by 90
  22. 2020RobBERT: a Dutch RoBERTa-based Language Modelcited by 88
  23. 2016Quasi-Recurrent Neural Networkscited by 87
  24. 2020End-to-End Slot Alignment and Recognition for Cross-Lingual NLUcited by 82
  25. 2016Neural Machine Translation in Linear Timecited by 65
  26. 2021Common Sense Beyond English: Evaluating and Improving Multilingual Language Models for Commonsense Reasoningcited by 64
  27. 2018Zero-shot User Intent Detection via Capsule Neural Networkscited by 54
  28. 2019How Language-Neutral is Multilingual BERT?cited by 51
  29. 2019Cross-Lingual BERT Transformation for Zero-Shot Dependency Parsingcited by 46
  30. 2020Single-/Multi-Source Cross-Lingual NER via Teacher-Student Learning on Unlabeled Data in Target Languagecited by 37
  31. 2020XED: A Multilingual Dataset for Sentiment Analysis and Emotion Detectioncited by 31
  32. 2020Self-supervised Knowledge Triplet Learning for Zero-shot Question Answeringcited by 29
  33. 2019Is Multilingual BERT Fluent in Language Generation?cited by 28
  34. 2020Exploring Cross-sentence Contexts for Named Entity Recognition with BERTcited by 27
  35. 2018Multilingual End-to-End Speech Recognition with A Single Transformer on Low-Resource Languagescited by 26
  36. 2019Multi-lingual Intent Detection and Slot Filling in a Joint BERT-based Modelcited by 24
  37. 2018A Comparison of Transformer and Recurrent Neural Networks on Multilingual Neural Machine Translationcited by 23
  38. 2020Cross-lingual Zero- and Few-shot Hate Speech Detection Utilising Frozen Transformer Language Models and AXELcited by 23

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 1.

1 paper.

  1. 2023Test-Time Adaptation with Perturbation Consistency Learningcited by 0