mlsanityData to 7 Oct 2026

GRASP LoRA: GRPO Guided Adapter Sparsity Policy for Cross Lingual Transfer

arXiv 2601.06702 ↗First posted 10 Jan 2026cs.CLcited by 0

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Parameter efficient fine tuning is a way to adapt LLMs to new languages when compute or data are limited, yet adapter pipelines usually choose a global prune ratio by grid search. This practice is computationally expensive and development set intensive, since it repeats training, freezes sparsity, and misses fractional optima. We introduce GRASP LoRA (GRPO Guided Adapter Sparsity Policy), which treats global sparsity as a learnable control variable. A GRPO controller interleaves with training, periodically probing candidate prune ratios on a small micro development set and updating a single global prune ratio online from its reward signal. It operates on merged source and target LoRA adapters on a frozen backbone and replaces grid search with one controller run that learns a prune ratio, followed by a single final merge and prune fine tuning run with pruning fixed to that ratio. On cross lingual transfer from English into Arabic and Chinese, including XL-Sum summarization and MLQA extractive question answering with Llama 3 8B, GRASP LoRA improves semantic faithfulness, content coverage, and answer quality over strong target only and merge and prune baselines. It reduces end to end runtime by multiple times relative to grid search, lowers reliance on large development sets, and makes adapter reuse practical for low resource deployment.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 21 of its 28 references that are in the map. First posted on arXiv 10 Jan 2026.

Builds on

21 of 28 references.

References · 21

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  3. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  4. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  5. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,234
  6. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,090
  7. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,028
  8. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
  9. 2020AdapterFusion: Non-Destructive Task Composition for Transfer Learningcited by 780
  10. 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 443
  11. 2020AdapterHub: A Framework for Adapting Transformerscited by 441
  12. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 426
9 more
  1. 2018AMC: AutoML for Model Compression and Acceleration on Mobile Devicescited by 375
  2. 2023LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Compositioncited by 359
  3. 2021XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languagescited by 346
  4. 2019MLQA: Evaluating Cross-lingual Extractive Question Answeringcited by 339
  5. 2025CPPO: Accelerating the Training of Group Relative Policy Optimization-Based Reasoning Modelscited by 93
  6. 2025Scaf-GRPO: Scaffolded Group Relative Policy Optimization for Enhancing LLM Reasoningcited by 48
  7. 2024LoRA-drop: Efficient LoRA Parameter Pruning based on Output Evaluationcited by 38
  8. 2025Adaptive LoRA Merge with Parameter Pruning for Low-Resource Generationcited by 5
  9. 2025RAP: Runtime Adaptive Pruning for LLM Inferencecited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026GPart: End-to-End Isometric Fine-Tuning via Global Parameter Partitioning
  2. 2024LoRA-XS: Low-Rank Adaptation with Extremely Small Number of Parameters
  3. 2026Sparsity-Aware Low-Rank Representation for Efficient Fine-Tuning of Large Language Models
  4. 2025Cross-LoRA: A Data-Free LoRA Transfer Framework across Heterogeneous LLMs
  5. 2025Dual LoRA: Enhancing LoRA with Magnitude and Direction Updates
  6. 2026Strategic Over-Parameterization for Generalizable Low-Rank Adaptation
  7. 2025Rethinking Parameter Sharing for LLM Fine-Tuning with Multiple LoRAs
  8. 2025CLoQ: Enhancing Fine-Tuning of Quantized LLMs via Calibrated LoRA Initialization