mlsanityData to 14 Oct 2026

Reclaiming Lost Text Layers for Source-Free Cross-Domain Few-Shot Learning

arXiv 2603.05235 ↗First posted 5 Mar 2026cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

Source-Free Cross-Domain Few-Shot Learning (SF-CDFSL) focuses on fine-tuning with limited training data from target domains (e.g., medical or satellite images), where CLIP has recently shown promising results due to its generalizability to downstream tasks. Current works indicate CLIP's text encoder is more suitable for cross-domain tasks, however, we find that removing certain middle layers of the text encoder can effectively improve performance in SF-CDFSL, which we call the Lost Layers. In this paper, we delve into this phenomenon for a deeper understanding. We discover that instead of being harmful for the SF-CDFSL task, the information in these layers is actually beneficial, but visual gaps prevent this useful information from being fully utilized, making these layers seem redundant. Based on this understanding, unlike current works that simply remove these layers, we propose a method to teachs the model to re-utilize information in these lost layers at both the layer and encoder levels, guiding the re-learning of the visual branch under domain shifts. Our approach effectively addresses the issue of underutilized information in the text encoder. Extensive experiments across various settings, backbones (CLIP, SigLip, PE-Core), and tasks (4 CDFSL datasets and 10 Meta-dataset datasets) demonstrate the effectiveness of our method. Code is available at https://github.com/zhenyuZ-HUST/CVPR26-VtT.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 73 of its 88 references that are in the map. First posted on arXiv 5 Mar 2026.

Builds on

73 of 88 references.

References · 73

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  5. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  6. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
  7. 2021Learning to Prompt for Vision-Language Modelscited by 2,281
  8. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,850
  9. 2019Similarity of Neural Network Representations Revisitedcited by 1,755
  10. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,496
  11. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,436
  12. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,433
38 more
  1. 2017Deep Hashing Network for Unsupervised Domain Adaptationcited by 1,188
  2. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
  3. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapterscited by 1,026
  4. 2017ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseasescited by 972
  5. 2021Robust fine-tuning of zero-shot modelscited by 797
  6. 2022MaPLe: Multi-modal Prompt Learningcited by 687
  7. 2019Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)cited by 669
  8. 2021Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modelingcited by 431
  9. 2024ShortGPT: Layers in Large Language Models are More Redundant Than You Expectcited by 383
  10. 2022Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classificationcited by 362
  11. 2023AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuningcited by 357
  12. 2022Prompt-aligned Gradient for Prompt Tuningcited by 341
  13. 2025Perception Encoder: The best visual embeddings are not at the output of the networkcited by 326
  14. 2019Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examplescited by 315
  15. 2023Visual-Language Prompt Tuning with Knowledge-guided Context Optimizationcited by 308
  16. 2023Self-regulating Prompts: Foundational Model Adaptation without Forgettingcited by 272
  17. 2022DyLoRA: Parameter Efficient Tuning of Pre-trained Models using Dynamic Search-Free Low-Rank Adaptationcited by 233
  18. 2016Using Deep Learning for Image-Based Plant Disease Detectioncited by 219
  19. 2014Recurrent Neural Network Regularizationcited by 189
  20. 2024The Remarkable Robustness of LLMs: Stages of Inference?cited by 146
  21. 2022Pushing the Limits of Simple Pipelines for Few-Shot Learning: External Data and Fine-Tuning Make a Differencecited by 126
  22. 2024Robust CLIP: Unsupervised Adversarial Fine-Tuning of Vision Embeddings for Robust Large Vision-Language Modelscited by 120
  23. 2022Understanding Zero-Shot Adversarial Robustness for Large-Scale Modelscited by 117
  24. 2021Understanding and Improving Early Stopping for Learning with Noisy Labelscited by 113
  25. 2024Low-Rank Few-Shot Adaptation of Vision-Language Modelscited by 104
  26. 2023One-for-All: Generalized LoRA for Parameter-Efficient Fine-tuningcited by 85
  27. 2023TCP:Textual-based Class-aware Prompt tuning for Visual-Language Modelcited by 76
  28. 2023Delta-LoRA: Fine-Tuning High-Rank Parameters with the Delta of Low-Rank Matricescited by 74
  29. 2024PromptKD: Unsupervised Prompt Distillation for Vision-Language Modelscited by 72
  30. 2021Cross-Domain Few-Shot Classification via Adversarial Task Augmentationcited by 63
  31. 2024LP++: A Surprisingly Strong Linear Probe for Few-Shot CLIPcited by 60
  32. 2023PromptStyler: Prompt-driven Style Generation for Source-free Domain Generalizationcited by 59
  33. 2023StyleAdv: Meta Style Adversarial Training for Cross-Domain Few-Shot Learningcited by 53
  34. 2021Boosting the Generalization Capability in Cross-Domain Few-shot Learning via Noise-enhanced Supervised Autoencodercited by 51
  35. 2023Prompt-based Distribution Alignment for Unsupervised Domain Adaptationcited by 44
  36. 2025Cross the Gap: Exposing the Intra-modal Misalignment in CLIP via Modality Inversioncited by 43
  37. 2022Adversarial Feature Augmentation for Cross-domain Few-shot Classificationcited by 36
  38. 2025DynaPrompt: Dynamic Test-Time Prompt Tuningcited by 31

50 most cited of 73 All 73 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026Interpretable Cross-Domain Few-Shot Learning with Rectified Target-Domain Local Alignmentcited by 4
  2. 2026Remedying Target-Domain Astigmatism for Cross-Domain Few-Shot Object Detectioncited by 2
  3. 2026Addressing Exacerbated Attention Sink for Source-Free Cross-Domain Few-Shot Learningcited by 2
  4. 2026Bridging Day and Night: Unsupervised Cross-Domain Re-Identification with Synergistic Prompt and Prototype Learningcited by 0
  5. 2026When Semantics Saturate or Emerge: Adaptation-Conditional Semantic Utility in Source-Free Cross-Domain Few-Shot Learningcited by 0