mlsanityData to 14 Oct 2026

Semantic Library Adaptation: LoRA Retrieval and Fusion for Open-Vocabulary Semantic Segmentation

arXiv 2503.21780 ↗First posted 27 Mar 2025cs.CVcited by 7

Not on the map. Why?

Abstract (arXiv)

Open-vocabulary semantic segmentation models associate vision and text to label pixels from an undefined set of classes using textual queries, providing versatile performance on novel datasets. However, large shifts between training and test domains degrade their performance, requiring fine-tuning for effective real-world applications. We introduce Semantic Library Adaptation (SemLA), a novel framework for training-free, test-time domain adaptation. SemLA leverages a library of LoRA-based adapters indexed with CLIP embeddings, dynamically merging the most relevant adapters based on proximity to the target domain in the embedding space. This approach constructs an ad-hoc model tailored to each specific input without additional training. Our method scales efficiently, enhances explainability by tracking adapter contributions, and inherently protects data privacy, making it ideal for sensitive applications. Comprehensive experiments on a 20-domain benchmark built over 10 standard datasets demonstrate SemLA's superior adaptability and performance across diverse settings, establishing a new standard in domain adaptation for open-vocabulary semantic segmentation.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 61 of its 88 references that are in the map. First posted on arXiv 27 Mar 2025.

Builds on

61 of 88 references.

References · 61

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  4. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  5. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  6. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  7. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
  8. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,223
  9. 2015Domain-Adversarial Training of Neural Networkscited by 3,044
  10. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
  11. 2016Pyramid Scene Parsing Networkcited by 2,479
  12. 2017Non-local Neural Networkscited by 2,293
38 more
  1. 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,487
  2. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,396
  3. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,288
  4. 2017CyCADA: Cycle-Consistent Adversarial Domain Adaptationcited by 1,082
  5. 2018Dual Attention Network for Scene Segmentationcited by 1,003
  6. 2016Playing for Data: Ground Truth from Computer Gamescited by 818
  7. 2021Robust fine-tuning of zero-shot modelscited by 797
  8. 2018BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentationcited by 645
  9. 2019Segmentation Transformer: Object-Contextual Representations for Semantic Segmentationcited by 576
  10. 2020FDA: Fourier Domain Adaptation for Semantic Segmentationcited by 541
  11. 2019Confidence Regularized Self-Trainingcited by 419
  12. 2023LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Compositioncited by 364
  13. 2019Bidirectional Learning for Domain Adaptation of Semantic Segmentationcited by 308
  14. 2021DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentationcited by 300
  15. 2021Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentationcited by 268
  16. 2023CAT-Seg: Cost Aggregation for Open-Vocabulary Semantic Segmentationcited by 219
  17. 2022Patching open-vocabulary models by interpolating weightscited by 190
  18. 2018Progressive Feature Alignment for Unsupervised Domain Adaptationcited by 183
  19. 2021Source-Free Domain Adaptation for Semantic Segmentationcited by 168
  20. 2019Gated-SCNN: Gated Shape CNNs for Semantic Segmentationcited by 166
  21. 2019Category Anchor-Guided Unsupervised Domain Adaptation for Semantic Segmentationcited by 151
  22. 2019Expectation-Maximization Attention Networks for Semantic Segmentationcited by 147
  23. 2022SHIFT: A Synthetic Driving Dataset for Continuous Multi-Task Domain Adaptationcited by 146
  24. 2020Classes Matter: A Fine-grained Adversarial Approach to Cross-domain Semantic Segmentationcited by 142
  25. 2019Boundary-Aware Feature Propagation for Scene Segmentationcited by 107
  26. 2018DCAN: Dual Channel-wise Alignment Networks for Unsupervised Scene Adaptationcited by 101
  27. 2023SED: A Simple Encoder-Decoder for Open-Vocabulary Semantic Segmentationcited by 97
  28. 2024Model Stock: All we need is just a few fine-tuned modelscited by 92
  29. 2023AdapterSoup: Weight Averaging to Improve Generalization of Pretrained Language Modelscited by 84
  30. 2024LoraRetriever: Input-Aware LoRA Retrieval and Composition for Mixed Tasks in the Wildcited by 68
  31. 2020Batch Normalization Embeddings for Deep Domain Generalizationcited by 57
  32. 2017Incremental Adversarial Domain Adaptation for Continually Changing Environmentscited by 52
  33. 2019Adaptive Context Network for Scene Parsingcited by 51
  34. 2015Semantic Image Segmentation with Task-Specific Edge Detection Using CNNs and a Discriminatively Trained Domain Transformcited by 46
  35. 2019ACE: Adapting to Changing Environments for Semantic Segmentationcited by 44
  36. 2020Unsupervised Model Adaptation for Continual Semantic Segmentationcited by 42
  37. 2020Gradient Regularized Contrastive Learning for Continual Domain Adaptationcited by 35
  38. 2024MUSES: The Multi-Sensor Semantic Perception Dataset for Driving under Uncertaintycited by 32

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 3.

7 papers.

  1. 2025HiLoRA: Adaptive Hierarchical LoRA Routing for Training-Free Domain Generalizationcited by 4
  2. 2026Diffusion Model as a Generalist Segmentation Learnercited by 4
  3. 2025Semantic-guided LoRA Parameters Generationcited by 1
  4. 2025Improving Visual Discriminability of CLIP for Training-Free Open-Vocabulary Semantic Segmentationcited by 1
  5. 2025CARLoS: Retrieval via Concise Assessment Representation of LoRAs at Scalecited by 0
  6. 2026Preference-Guided Adaptation for Open-Vocabulary Semantic Segmentation via Prompt Disagreementcited by 0
  7. 2026When to Adapt: Multi-Signal Domain Shift Detection for Efficient Training-Free Adaptation in Open-Vocabulary Segmentationcited by 0