Topology-Aware CLIP Few-Shot Learning
On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP
Abstract (arXiv)
Efficiently adapting large Vision-Language Models (VLMs) like CLIP for few-shot learning poses challenges in balancing pre-trained knowledge retention and task-specific adaptation. Existing methods often overlook valuable structural information within the VLM's latent space. We introduce a topology-aware tuning approach integrating Representation Topology Divergence (RTD) into the Task Residual (TR) framework. By explicitly aligning the topological structures of visual and text representations using a combined RTD and Cross-Entropy loss, while freezing base VLM encoders, our method enhances few-shot performance. We optimize only lightweight Task Residual parameters, effectively leveraging topological information. Across 6 diverse benchmark datasets, our approach demonstrates significant gains, achieving an average accuracy improvement of 1-2% over relevant baseline methods in few-shot settings. This work presents an effective strategy to boost VLM few-shot capabilities by incorporating topological alignment.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 20 of its 39 references that are in the map. First posted on arXiv 3 May 2025.
Builds on
20 of 39 references.
References · 20
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
- 2013Describing Textures in the Wildcited by 2,125
- 2013Fine-Grained Visual Classification of Aircraftcited by 1,726
- 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,496
- 2019Topology-Preserving Deep Image Segmentationcited by 198
- 2019A Topological Loss Function for Deep-Learning based Image Segmentation using Persistent Homologycited by 134
- 2019Topological Autoencoderscited by 127
- 2021Topological Graph Neural Networkscited by 87
- 2021Topology-Aware Segmentation Using Discrete Morse Theorycited by 75
- 2018Neural Persistence: A Complexity Measure for Deep Neural Networks Using Algebraic Topologycited by 73
- 2021Representation Topology Divergence: A Method for Comparing Neural Network Representationscited by 66
8 more
- 2019Learning metrics for persistence-based summaries and applications for graph classificationcited by 57
- 2021Manifold Topology Divergence: a Framework for Comparing Data Manifoldscited by 31
- 2023Learning Topology-Preserving Data Representationscited by 31
- 2020Uncovering the Topology of Time-Varying fMRI Data using Cubical Persistencecited by 26
- 2022Learning Probabilistic Topological Representations Using Discrete Morse Theorycited by 20
- 2023Curvature Filtrations for Graph Generative Model Evaluationcited by 19
- 2022ToDD: Topological Compound Fingerprinting in Computer-Aided Drug Discoverycited by 13
- 2023Improving Self-supervised Molecular Representation Learning using Persistent Homologycited by 6
Built on by
Citing papers per year: 2025: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025Preserve and Sculpt: Manifold-Aligned Fine-tuning of Vision-Language Models for Few-Shot Learning
- 2025Language-Aware Information Maximization for Transductive Few-Shot CLIP
- 2026Topology-Aware Layer Pruning for Large Vision-Language Models
- 2026TagaVLM: Topology-Aware Global Action Reasoning for Vision-Language Navigation
- 2026Topology-Aware Representation Alignment for Semi-Supervised Vision-Language Learning
- 2026Improving CLIP Adaptation by Breaking Tail Alignment for Source-Free Cross-Domain Few-Shot Learning
- 2026TASSO: TAsk-Specific Subspace Optimization for Continual Learning of Vision-Language Models
- 2024Fully Fine-tuned CLIP Models are Efficient Few-Shot Learners