# PLoP: Precise LoRA Placement for Efficient Finetuning of Large Models

arXiv: https://arxiv.org/abs/2506.20629 · HTML: https://mlsanity.com/paper/2506.20629/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 39 references that are in the map.
- First posted on arXiv 25 Jun 2025.

## Builds on (most cited first, 27 of 27)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge](https://mlsanity.com/paper/1811.00937/) (2018)
- [Towards a Unified View of Parameter-Efficient Transfer Learning](https://mlsanity.com/paper/2110.04366/) (2021)
- [Adversarial NLI: A New Benchmark for Natural Language Understanding](https://mlsanity.com/paper/1910.14599/) (2019)
- [DoRA: Weight-Decomposed Low-Rank Adaptation](https://mlsanity.com/paper/2402.09353/) (2024)
- [MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models](https://mlsanity.com/paper/2309.12284/) (2023)
- [signSGD: Compressed Optimisation for Non-Convex Problems](https://mlsanity.com/paper/1802.04434/) (2018)
- [LoRA+: Efficient Low Rank Adaptation of Large Models](https://mlsanity.com/paper/2402.12354/) (2024)
- [AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning](https://mlsanity.com/paper/2303.10512/) (2023)
- [Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learning](https://mlsanity.com/paper/2405.00451/) (2024)
- [Feature Learning in Infinite-Width Neural Networks](https://mlsanity.com/paper/2011.14522/) (2020)
- [Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivation](https://mlsanity.com/paper/1902.04760/) (2019)
- [HydraLoRA: An Asymmetric LoRA Architecture for Efficient Fine-Tuning](https://mlsanity.com/paper/2404.19245/) (2024)
- [On the Impact of the Activation Function on Deep Neural Networks Training](https://mlsanity.com/paper/1902.06853/) (2019)
- [Cross-Attention is All You Need: Adapting Pretrained Transformers for Machine Translation](https://mlsanity.com/paper/2104.08771/) (2021)
- [Sensitivity-Aware Visual Parameter-Efficient Fine-Tuning](https://mlsanity.com/paper/2303.08566/) (2023)
- [The Impact of Initialization on LoRA Finetuning Dynamics](https://mlsanity.com/paper/2406.08447/) (2024)
- [Gradient-based Parameter Selection for Efficient Fine-Tuning](https://mlsanity.com/paper/2312.10136/) (2023)
- [Feature Learning and Signal Propagation in Deep Neural Networks](https://mlsanity.com/paper/2110.11749/) (2021)
- [A Note on LoRA](https://mlsanity.com/paper/2404.05086/) (2024)
- [Understanding and Minimising Outlier Features in Neural Network Training](https://mlsanity.com/paper/2405.19279/) (2024)

## Built on by (most cited first, 14 of 14)

- [Learning Rate Matters: Vanilla LoRA May Suffice for LLM Fine-tuning](https://mlsanity.com/paper/2602.04998/) (2026)
- [Learning Rate Scaling across LoRA Ranks and Transfer to Full Finetuning](https://mlsanity.com/paper/2602.06204/) (2026)
- [PoLoRA: A Preconditioned Orthogonalized LoRA Optimizer](https://mlsanity.com/paper/2607.17620/) (2026)
- [Rethinking Adapter Placement: A Dominant Adaptation Module Perspective](https://mlsanity.com/paper/2605.06183/) (2026)
- [Non-vacuous Generalization Bounds for Reinforcement Learning with Verifiable Rewards](https://mlsanity.com/paper/2607.14506/) (2026)
- [Training-Free versus Training-Based Intent Classification in LLMs: Accuracy, Robustness, and Failure Modes](https://mlsanity.com/paper/2608.02415/) (2026)
- [α-LoRA: Effective Fine-Tuning via Base Model Rescaling](https://mlsanity.com/paper/2510.21345/) (2025)
- [Symmetry Defeats Auditing](https://mlsanity.com/paper/2605.27836/) (2026)
- [Data-Efficient Adaptation of LLMs via Attention Head Reweighting](https://mlsanity.com/paper/2607.13425/) (2026)
- [Test-Time Registers as Global Priors for Tokenized Image Generation](https://mlsanity.com/paper/2607.16824/) (2026)
- [ACE: Adapter Consolidation across Experts for Parameter-Efficient Fine-Tuning of MoE LLMs](https://mlsanity.com/paper/2609.06072/) (2026)
- [What Does Layer-Importance Reveal About Transformers and State-Space Models?](https://mlsanity.com/paper/2609.16537/) (2026)
- [Reassessing Global Gradient-Norm Imbalance in BLIP Fine-Tuning Across Physical Domains](https://mlsanity.com/paper/2609.23655/) (2026)
- [ReGRA: Residual-Gradient Adaptation for Complementary Low-Rank Fine-Tuning](https://mlsanity.com/paper/2610.19700/) (2026)

## Abstract (arXiv)

Low-Rank Adaptation (LoRA) is a widely used finetuning method for large models. Its small memory footprint allows practitioners to adapt large models to specific tasks at a fraction of the cost of full finetuning. Different modifications have been proposed to enhance its efficiency by, for example, setting the learning rate, the rank, and the initialization. Another improvement axis is adapter placement strategy: when using LoRA, practitioners usually pick module types to adapt with LoRA, such as Query and Key modules. Few works have studied the problem of adapter placement, with nonconclusive results: original LoRA paper suggested placing adapters in attention modules, while other works suggested placing them in the MLP modules. Through an intuitive theoretical analysis, we introduce PLoP (Precise LoRA Placement), a lightweight method that allows automatic identification of module types where LoRA adapters should be placed, given a pretrained model and a finetuning task. We demonstrate that PLoP consistently outperforms, and in the worst case competes, with commonly used placement strategies through comprehensive experiments on supervised finetuning and reinforcement learning for reasoning.