mlsanityData to 14 Oct 2026

Low-Rank Adaptation for Multilingual Summarization: An Empirical Study

arXiv 2311.08572 ↗First posted 14 Nov 2023cs.CL cs.AI cs.LGcited by 13

Not on the map. Why?

Abstract (arXiv)

Although the advancements of pre-trained Large Language Models have significantly accelerated recent progress in NLP, their ever-increasing size poses significant challenges for conventional fine-tuning, especially in memory-intensive tasks. We investigate the potential of Parameter-Efficient Fine-Tuning, focusing on Low-Rank Adaptation (LoRA), in the domain of multilingual summarization, a task that is both challenging (due to typically long inputs), and relatively unexplored. We conduct an extensive study across different data availability scenarios, including high- and low-data settings, and cross-lingual transfer, leveraging models of different sizes. Our findings reveal that LoRA is competitive with full fine-tuning when trained with high quantities of data, and excels in low-data scenarios and cross-lingual transfer. We also study different strategies for few-shot cross-lingual transfer, finding that continued LoRA tuning outperforms full fine-tuning and the dynamic composition of language-specific LoRA modules.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 34 of its 49 references that are in the map. First posted on arXiv 14 Nov 2023.

Builds on

34 of 49 references.

References · 34

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  4. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  5. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  6. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  7. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  8. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  9. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  10. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  11. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  12. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
22 more
  1. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,729
  2. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 915
  3. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 872
  4. 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 807
  5. 2020AdapterFusion: Non-Destructive Task Composition for Transfer Learningcited by 788
  6. 2019On the Cross-lingual Transferability of Monolingual Representationscited by 578
  7. 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 445
  8. 2022Dataless Knowledge Fusion by Merging Weights of Language Modelscited by 387
  9. 2023LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Compositioncited by 364
  10. 2021XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languagescited by 347
  11. 2022Black-Box Tuning for Language-Model-as-a-Servicecited by 256
  12. 2019Cross-Lingual Ability of Multilingual BERT: An Empirical Studycited by 182
  13. 2021Composable Sparse Fine-Tuning for Cross-Lingual Transfercited by 144
  14. 2020NILE : Natural Language Inference with Faithful Natural Language Explanationscited by 96
  15. 2022Revisiting Parameter-Efficient Tuning: Are We Really There Yet?cited by 76
  16. 2022Overcoming Catastrophic Forgetting in Zero-Shot Cross-Lingual Generationcited by 62
  17. 2022mFACE: Multilingual Summarization with Factual Consistency Evaluationcited by 42
  18. 2023SEAHORSE: A Multilingual, Multifaceted Dataset for Summarization Evaluationcited by 42
  19. 2022Models and Datasets for Cross-Lingual Summarisationcited by 36
  20. 2022EntityCS: Improving Zero-Shot Cross-lingual Transfer with Entity-Centric Code Switchingcited by 8
  21. 2023WebIE: Faithful and Robust Information Extraction on the Webcited by 8
  22. 2023μPLAN: Summarizing using a Content Plan as Cross-Lingual Bridgecited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 6; 2026: 5.

13 papers.

Sort:

By area:

  1. 2024CALRec: Contrastive Alignment of Generative LLMs for Sequential Recommendationcited by 24
  2. 2025Magical: Medical Lay Language Generation via Semantic Invariance and Layperson-tailored Adaptationcited by 5
  3. 2025Document Intelligence in the Era of Large Language Models: A Surveycited by 5
  4. 2025Enhancing LLM Language Adaption through Cross-lingual In-Context Pre-trainingcited by 4
  5. 2024DLP-LoRA: Efficient Task-Specific LoRA Fusion with a Dynamic, Lightweight Plugin for Large Language Modelscited by 2
  6. 2025SECURA: Sigmoid-Enhanced CUR Decomposition with Uninterrupted Retention and Low-Rank Adaptation in Large Language Modelscited by 1
  7. 2025FALQON: Accelerating LoRA Fine-tuning with Low-Bit Floating-Point Arithmeticcited by 1
  8. 2026Multilinguality at the Edge: Developing Language Models for the Global Southcited by 1
  9. 2025WRAVAL -- WRiting Assist eVALuationcited by 0
  10. 2026Bootstrapping Embeddings for Low Resource Languagescited by 0
  11. 2026Cross-Lingual Transfer and Parameter-Efficient Adaptation in the Turkic Language Family: A Theoretical Framework for Low-Resource Language Modelscited by 0
  12. 2026Output Composability of QLoRA PEFT Modules for Plug-and-Play Attribute-Controlled Text Generationcited by 0
1 more
  1. 2026Which LoRA? An Empirical Study on the Effectiveness of LoRA Techniques During Multilingual Instruction Tuningcited by 0