# LoFA: Learning to Predict Personalized Priors for Fast Adaptation of Visual Generative Models

arXiv: https://arxiv.org/abs/2512.08785 · HTML: https://mlsanity.com/paper/2512.08785/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Dec 2025.

## Built on by (most cited first, 1 of 1)

- [Prompt2Effect: Training-Free Image-to-Video Model Specialization via LoRA Generation](https://mlsanity.com/paper/2606.13971/) (2026)

## Abstract (arXiv)

Personalizing visual generative models to meet specific user needs has gained increasing attention, yet current methods like Low-Rank Adaptation (LoRA) remain impractical due to their demand for task-specific data and lengthy optimization. While a few hypernetwork-based approaches attempt to predict adaptation weights directly, they struggle to map fine-grained user prompts to complex LoRA distributions, limiting their practical applicability. To bridge this gap, we propose LoFA, a general framework that efficiently predicts personalized priors for fast model adaptation. We first identify a key property of LoRA: structured distribution patterns emerge in the relative changes between LoRA and base model parameters. Building on this, we design a two-stage hypernetwork: first predicting relative distribution patterns that capture key adaptation regions, then using these to guide final LoRA weight prediction. Extensive experiments demonstrate that our method consistently predicts high-quality personalized priors within seconds, across multiple tasks and user prompts, even outperforming conventional LoRA that requires hours of processing. Project page: https://jaeger416.github.io/lofa/.