# WaRA: Wavelet Low-Rank Adaptation for Medical Image Classification

arXiv: https://arxiv.org/abs/2506.24092 · HTML: https://mlsanity.com/paper/2506.24092/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 28 references that are in the map.
- First posted on arXiv 25 Jun 2025.

## Builds on (most cited first, 22 of 22)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Visual Prompt Tuning](https://mlsanity.com/paper/2203.12119/) (2022)
- [AdapterFusion: Non-Destructive Task Composition for Transfer Learning](https://mlsanity.com/paper/2005.00247/) (2020)
- [Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)](https://mlsanity.com/paper/1902.03368/) (2019)
- [Parameter-Efficient Transfer Learning with Diff Pruning](https://mlsanity.com/paper/2012.07463/) (2020)
- [Scaling Down to Scale Up: A Guide to Parameter-Efficient Fine-Tuning](https://mlsanity.com/paper/2303.15647/) (2023)
- [On the Effectiveness of Parameter-Efficient Fine-Tuning](https://mlsanity.com/paper/2211.15583/) (2022)
- [Three things everyone should know about Vision Transformers](https://mlsanity.com/paper/2203.09795/) (2022)
- [Cross-domain Few-shot Learning with Task-specific Adapters](https://mlsanity.com/paper/2107.00358/) (2021)
- [Wave-ViT: Unifying Wavelet and Transformers for Visual Representation Learning](https://mlsanity.com/paper/2207.04978/) (2022)
- [LoRA-XS: Low-Rank Adaptation with Extremely Small Number of Parameters](https://mlsanity.com/paper/2405.17604/) (2024)
- [Parameter-Efficient Fine-Tuning with Discrete Fourier Transform](https://mlsanity.com/paper/2405.03003/) (2024)
- [Parameter-Efficient Fine-Tuning for Medical Image Analysis: The Missed Opportunity](https://mlsanity.com/paper/2305.08252/) (2023)
- [HiFuse: Hierarchical Multi-Scale Feature Fusion Network for Medical Image Classification](https://mlsanity.com/paper/2209.10218/) (2022)
- [Low-Rank Mixture-of-Experts for Continual Medical Image Segmentation](https://mlsanity.com/paper/2406.13583/) (2024)
- [FouRA: Fourier Low Rank Adaptation](https://mlsanity.com/paper/2406.08798/) (2024)
- [ShareLoRA: Parameter Efficient and Robust Large Language Model Fine-tuning via Shared Low-Rank Adaptation](https://mlsanity.com/paper/2406.10785/) (2024)
- [Wavelets Are All You Need for Autoregressive Image Generation](https://mlsanity.com/paper/2406.19997/) (2024)
- [Exploring Sparsity for Parameter Efficient Fine Tuning Using Wavelets for Vision](https://mlsanity.com/paper/2505.12532/) (2025)
- [LoCA: Location-Aware Cosine Adaptation for Parameter-Efficient Fine-Tuning](https://mlsanity.com/paper/2502.06820/) (2025)

## Built on by (most cited first, 1 of 1)

- [Frequency-Aware Continual Learning for Smart Contract Vulnerability Detection with Large Language Models](https://mlsanity.com/paper/2608.19680/) (2026)

## Abstract (arXiv)

Adapting large pretrained vision models to medical image classification is often limited by memory, computation, and task-specific specializations. Parameter-efficient fine-tuning (PEFT) methods like LoRA reduce this cost by learning low-rank updates, but operating directly in feature space can struggle to capture the localized, multi-scale features common in medical imaging. We propose WaRA, a wavelet-structured adaptation module that performs low-rank adaptation in a wavelet domain. WaRA reshapes patch tokens into a spatial grid, applies a fixed discrete wavelet transform, updates subband coefficients using a shared low-rank adapter, and reconstructs the additive update through an inverse wavelet transform. This design provides a compact trainable interface while biasing the update toward both coarse structure and fine detail. For extremely low-resource settings, we introduce Tiny-WaRA, which further reduces trainable parameters by learning only a small set of coefficients in a fixed basis derived from the pretrained weights through a truncated SVD. Experiments on medical image classification across four modalities and datasets demonstrate that WaRA consistently improves performance over strong PEFT baselines, while retaining a favorable efficiency profile. Our code is publicly available at https://github.com/moeinheidari7829/WaRA.