mlsanityData to 7 Oct 2026

MultiFiT: Efficient Multi-lingual Language Model Fine-tuning

arXiv 1909.04761 ↗First posted 10 Sep 2019cs.CL cs.LGcited by 31

Not on the map. Why?

Abstract (arXiv)

Pretrained language models are promising particularly for low-resource languages as they only require unlabelled data. However, training existing models requires huge amounts of compute, while pretrained cross-lingual models often underperform on low-resource languages. We propose Multi-lingual language model Fine-Tuning (MultiFiT) to enable practitioners to train and fine-tune language models efficiently in their own language. In addition, we propose a zero-shot method using an existing pretrained cross-lingual model. We evaluate our methods on two widely used cross-lingual classification datasets where they outperform models pretrained on orders of magnitude more data and compute. We release all models and code.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 15; 2022: 2; 2023: 6; 2024: 3; 2025: 0; 2026: 5.

31 papers.

Sort:

By area:

  1. 2021MultiEURLEX -- A multi-lingual and multi-label legal document classification dataset for zero-shot cross-lingual transfercited by 89
  2. 2026Being-H0.5: Scaling Human-Centric Robot Learning for Cross-Embodiment Generalizationcited by 61
  3. 2021A Primer on Pretrained Multilingual Language Modelscited by 49
  4. 2022Improved Multi-label Classification under Temporal Concept Drift: Rethinking Group-Robust Algorithms in a Label-Wise Settingcited by 23
  5. 2023Taxi1500: A Multilingual Dataset for Text Classification in 1500 Languagescited by 23
  6. 2021Analysing The Impact Of Linguistic Features On Cross-Lingual Transfercited by 16
  7. 2021Everything Is All It Takes: A Multipronged Strategy for Zero-Shot Cross-Lingual Information Extractioncited by 16
  8. 2021Multilingual and cross-lingual document classification: A meta-learning approachcited by 14
  9. 2024Challenges in Adapting Multilingual LLMs to Low-Resource Languages using LoRA PEFT Tuningcited by 14
  10. 2024TeenyTinyLlama: open-source tiny language models trained in Brazilian Portuguesecited by 12
  11. 2021Can Character-based Language Models Improve Downstream Task Performance in Low-Resource and Noisy Language Scenarios?cited by 11
  12. 2024BayLing 2: A Multilingual Large Language Model with Efficient Language Alignmentcited by 8
19 more
  1. 2021Fake News Detection System using XLNet model with Topic Distributions: CONSTRAINT@AAAI2021 Shared Taskcited by 6
  2. 2021Lightweight Cross-Lingual Sentence Representation Learningcited by 5
  3. 2021Rumour Detection via Zero-shot Cross-lingual Transfer Learningcited by 5
  4. 2021LegaLMFiT: Efficient Short Legal Text Classification with LSTM Language Model Pre-Trainingcited by 4
  5. 2022Multilingual Transformer Encoders: a Word-Level Task-Agnostic Evaluationcited by 4
  6. 2023Cross-lingual Transfer Can Worsen Bias in Sentiment Analysiscited by 3
  7. 2023Human-AI Collaboration in Thematic Analysis using ChatGPT: A User Study and Design Recommendationscited by 3
  8. 2021A Multilingual Bag-of-Entities Model for Zero-Shot Cross-Lingual Text Classificationcited by 2
  9. 2023Multilingual Multiword Expression Identification Using Lateral Inhibition and Domain Adaptationcited by 2
  10. 2026Improving Training Efficiency and Reducing Maintenance Costs via Language Specific Model Mergingcited by 2
  11. 2021Improved Multilingual Language Model Pretraining for Social Media Text via Translation Pair Predictioncited by 1
  12. 2021Generalized Funnelling: Ensemble Learning and Heterogeneous Document Embeddings for Cross-Lingual Text Classificationcited by 1
  13. 2021Improved Customer Transaction Classification using Semi-Supervised Knowledge Distillationcited by 0
  14. 2021Priberam Labs at the NTCIR-15 SHINRA2020-ML: Classification Taskcited by 0
  15. 2023Gradient Sparsification For Masked Fine-Tuning of Transformerscited by 0
  16. 2023Zero-shot Cross-lingual Transfer without Parallel Corpuscited by 0
  17. 2026Effective vocabulary expansion of multilingual language models for extremely low-resource languagescited by 0
  18. 2026MzansiText and MzansiLM: An Open Corpus and Decoder-Only Language Model for South African Languagescited by 0
  19. 2026Modular Monolingual Adaptation using Pretrained Language Modelscited by 0