mlsanityData to 7 Oct 2026

Unlocking Parameter-Efficient Fine-Tuning for Low-Resource Language Translation

arXiv 2404.04212 ↗First posted 5 Apr 2024cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

Parameter-efficient fine-tuning (PEFT) methods are increasingly vital in adapting large-scale pre-trained language models for diverse tasks, offering a balance between adaptability and computational efficiency. They are important in Low-Resource Language (LRL) Neural Machine Translation (NMT) to enhance translation accuracy with minimal resources. However, their practical effectiveness varies significantly across different languages. We conducted comprehensive empirical experiments with varying LRL domains and sizes to evaluate the performance of 8 PEFT methods with in total of 15 architectures using the SacreBLEU score. We showed that 6 PEFT architectures outperform the baseline for both in-domain and out-domain tests and the Houlsby+Inversion adapter has the best performance overall, proving the effectiveness of PEFT methods.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 26 of its 34 references that are in the map. First posted on arXiv 5 Apr 2024.

Builds on

26 of 34 references.

References · 26

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  3. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  4. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  5. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
  6. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
  7. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 863
  8. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 789
  9. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
  10. 2020Beyond English-Centric Multilingual Machine Translationcited by 559
  11. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 556
  12. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
14 more
  1. 2021Compacter: Efficient Low-Rank Hypercomplex Adapter Layerscited by 458
  2. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
  3. 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 443
  4. 2020AdapterHub: A Framework for Adapting Transformerscited by 441
  5. 2019Simple, Scalable Adaptation for Neural Machine Translationcited by 239
  6. 2021UniPELT: A Unified Framework for Parameter-Efficient Language Model Tuningcited by 162
  7. 2023Don't Trust ChatGPT when Your Question is not in English: A Study of Multilingual Abilities and Types of LLMscited by 103
  8. 2023Adapters: A Unified Library for Parameter-Efficient and Modular Transfer Learningcited by 53
  9. 2017Regularization techniques for fine-tuning in neural machine translationcited by 27
  10. 2022IndicMT Eval: A Dataset to Meta-Evaluate Machine Translation metrics for Indian Languagescited by 27
  11. 2022Pre-Trained Multilingual Sequence-to-Sequence Models: A Hope for Low-Resource Language Translation?cited by 24
  12. 2021Multilingual Unsupervised Neural Machine Translation with Denoising Adapterscited by 22
  13. 2020Data Augmentation and Terminology Integration for Domain-Specific Sinhala-English-Tamil Statistical Machine Translationcited by 16
  14. 2022When does Parameter-Efficient Transfer Learning Work for Machine Translation?cited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025Minifinetuning: Low-Data Generation Domain Adaptation through Corrective Self-Distillationcited by 1
  2. 2025Uncovering Cross-Linguistic Disparities in LLMs using Sparse Autoencoderscited by 1
  3. 2025NagaNLP: Bootstrapping NLP for Low-Resource Nagamese Creole with Human-in-the-Loop Synthetic Datacited by 1
  4. 2026Efficient Low-Resource Language Adaptation via Multi-Source Dynamic Logit Fusioncited by 0