# Complementary Subspace Low-Rank Adaptation of Vision-Language Models for Few-Shot Classification

arXiv: https://arxiv.org/abs/2501.15040 · HTML: https://mlsanity.com/paper/2501.15040/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 5 of its 77 references that are in the map.
- First posted on arXiv 25 Jan 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Prompt learning and adaptation of CLIP](https://mlsanity.com/mllm/prompt-learning-and-adaptation-of-clip/)

## Builds on (most cited first, 5 of 5)

- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Fine-Grained Visual Classification of Aircraft](https://mlsanity.com/paper/1306.5151/) (2013)
- [Orthogonal Subspace Learning for Language Model Continual Learning](https://mlsanity.com/paper/2310.14152/) (2023)
- [PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models](https://mlsanity.com/paper/2404.02948/) (2024)
- [LoRA-Pro: Are Low-Rank Adapters Properly Optimized?](https://mlsanity.com/paper/2407.18242/) (2024)

## Built on by (most cited first, 2 of 2)

- [Subspace Alignment for Vision-Language Model Test-time Adaptation](https://mlsanity.com/paper/2601.08139/) (2026)
- [MMLoP: Multi-Modal Low-Rank Prompting for Efficient Vision-Language Adaptation](https://mlsanity.com/paper/2602.21397/) (2026)

## Abstract (arXiv)

Vision language model (VLM) has been designed for large scale image-text alignment as a pretrained foundation model. For downstream few shot classification tasks, parameter efficient fine-tuning (PEFT) VLM has gained much popularity in the computer vision community. PEFT methods like prompt tuning and linear adapter have been studied for fine-tuning VLM while low rank adaptation (LoRA) algorithm has rarely been considered for few shot fine-tuning VLM. The main obstacle to use LoRA for few shot fine-tuning is the catastrophic forgetting problem. Because the visual language alignment knowledge is important for the generality in few shot learning, whereas low rank adaptation interferes with the most informative direction of the pretrained weight matrix. We propose the complementary subspace low rank adaptation (Comp-LoRA) method to regularize the catastrophic forgetting problem in few shot VLM finetuning. In detail, we optimize the low rank matrix in the complementary subspace, thus preserving the general vision language alignment ability of VLM when learning the novel few shot information. We conduct comparison experiments of the proposed Comp-LoRA method and other PEFT methods on fine-tuning VLM for few shot classification. And we also present the suppression on the catastrophic forgetting problem of our proposed method against directly applying LoRA to VLM. The results show that the proposed method surpasses the baseline method by about +1.0% Top-1 accuracy and preserves the VLM zero-shot performance over the baseline method by about +1.3% Top-1 accuracy.