# YOLO-PEFT: Parameter-Efficient Fine-Tuning on YOLO Family

arXiv: https://arxiv.org/abs/2608.07051 · HTML: https://mlsanity.com/paper/2608.07051/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 37 of its 45 references that are in the map.
- First posted on arXiv 7 Aug 2026.

## Builds on (most cited first, 30 of 37)

- [Visual Prompt Tuning](https://mlsanity.com/paper/2203.12119/) (2022)
- [Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning](https://mlsanity.com/paper/2205.05638/) (2022)
- [Towards a Unified View of Parameter-Efficient Transfer Learning](https://mlsanity.com/paper/2110.04366/) (2021)
- [DoRA: Weight-Decomposed Low-Rank Adaptation](https://mlsanity.com/paper/2402.09353/) (2024)
- [AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition](https://mlsanity.com/paper/2205.13535/) (2022)
- [Learning multiple visual domains with residual adapters](https://mlsanity.com/paper/1705.08045/) (2017)
- [YOLO-World: Real-Time Open-Vocabulary Object Detection](https://mlsanity.com/paper/2401.17270/) (2024)
- [Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detection](https://mlsanity.com/paper/2006.04388/) (2020)
- [AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning](https://mlsanity.com/paper/2303.10512/) (2023)
- [VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Tasks](https://mlsanity.com/paper/2112.06825/) (2021)
- [YOLOv12: Attention-Centric Real-Time Object Detectors](https://mlsanity.com/paper/2502.12524/) (2025)
- [Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuning](https://mlsanity.com/paper/2210.08823/) (2022)
- [Convolutional Bypasses Are Better Vision Transformer Adapters](https://mlsanity.com/paper/2207.07039/) (2022)
- [Parameter-efficient Model Adaptation for Vision Transformers](https://mlsanity.com/paper/2203.16329/) (2022)
- [Sensitivity-Aware Visual Parameter-Efficient Fine-Tuning](https://mlsanity.com/paper/2303.08566/) (2023)
- [Conv-Adapter: Exploring Parameter Efficient Transfer Learning for ConvNets](https://mlsanity.com/paper/2208.07463/) (2022)
- [VMT-Adapter: Parameter-Efficient Transfer Learning for Multi-Task Dense Scene Understanding](https://mlsanity.com/paper/2312.08733/) (2023)
- [Pro-tuning: Unified Prompt Tuning for Vision Tasks](https://mlsanity.com/paper/2207.14381/) (2022)
- [AutoPEFT: Automatic Configuration Search for Parameter-Efficient Fine-Tuning](https://mlsanity.com/paper/2301.12132/) (2023)
- [Parameter-efficient is not sufficient: Exploring Parameter, Memory, and Time Efficient Adapter Tuning for Dense Predictions](https://mlsanity.com/paper/2306.09729/) (2023)
- [SCT: A Simple Baseline for Parameter-Efficient Fine-Tuning via Salient Channels](https://mlsanity.com/paper/2309.08513/) (2023)
- [Bridging The Gap between Low-rank and Orthogonal Adaptation via Householder Reflection Adaptation](https://mlsanity.com/paper/2405.17484/) (2024)
- [Source-Free Domain Adaptation for YOLO Object Detection](https://mlsanity.com/paper/2409.16538/) (2024)
- [DA-Ada: Learning Domain-Aware Adapter for Domain Adaptive Object Detection](https://mlsanity.com/paper/2410.09004/) (2024)
- [CoPEFT: Fast Adaptation Framework for Multi-Agent Collaborative Perception with Parameter-Efficient Fine-Tuning](https://mlsanity.com/paper/2502.10705/) (2025)
- [LORS: Low-rank Residual Structure for Parameter-Efficient Network Stacking](https://mlsanity.com/paper/2403.04303/) (2024)
- [SIA-OVD: Shape-Invariant Adapter for Bridging the Image-Region Gap in Open-Vocabulary Detection](https://mlsanity.com/paper/2410.05650/) (2024)
- [SpotPatch: Parameter-Efficient Transfer Learning for Mobile Object Detection](https://mlsanity.com/paper/2101.01260/) (2021)
- [Low-Rank Adaption on Transformer-based Oriented Object Detector for Satellite Onboard Processing of Remote Sensing Images](https://mlsanity.com/paper/2406.02385/) (2024)
- [PET-DINO: Unifying Visual Cues into Grounding DINO with Prompt-Enriched Training](https://mlsanity.com/paper/2604.00503/) (2026)

## Abstract (arXiv)

Generic parameter-efficient fine-tuning (PEFT) methods transferred from language models can fail silently on real-time detectors, whose heterogeneous operators and detection-specific components impose placement constraints absent from regular Transformer stacks. We propose YOLO-PEFT, a structure-aware framework that formulates adapter placement as an auditable constraint-planning problem. Given a detector graph, a PEFT request, and a resource budget, YOLO-PEFT assigns operator and semantic roles, evaluates explicit operator-validity, detector-semantic, graph-interface, and deployment predicates, records a reason code for each excluded module, and either emits a budgeted target-module plan or returns Refuse before training. Under the official VOC07+12 trainval-to-VOC07 test protocol, planner-selected RS-LoRA reaches 0.7138 and 0.7307 mAP50-95 on YOLO11s and YOLO12s, respectively, compared with 0.6428 and 0.6662 for Full-SFT. On RT-DETR-L, all seven evaluated LoRA-family configurations cross the predefined catastrophic threshold, supporting a calibrated Refuse-to-Full-SFT decision within the evaluated coverage. A controlled YOLO11 audit further shows that LoRA reduces peak training memory by 43.9 percent, although training takes 1.72 times longer. Within the evaluated detector families, placement policies, and calibration coverage, YOLO-PEFT replaces manual target-module trial and error with explicit, inspectable planning while preserving verified train-save-merge-export paths; refusal on unseen detector architectures remains an open validation problem. Project Page: github.com/Tencent/YOLO-Master