# Topology-Aware CLIP Few-Shot Learning

arXiv: https://arxiv.org/abs/2505.01694 · HTML: https://mlsanity.com/paper/2505.01694/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 20 of its 39 references that are in the map.
- First posted on arXiv 3 May 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Prompt learning and adaptation of CLIP](https://mlsanity.com/mllm/prompt-learning-and-adaptation-of-clip/)

## Builds on (most cited first, 20 of 20)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [Fine-Grained Visual Classification of Aircraft](https://mlsanity.com/paper/1306.5151/) (2013)
- [EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification](https://mlsanity.com/paper/1709.00029/) (2017)
- [Topology-Preserving Deep Image Segmentation](https://mlsanity.com/paper/1906.05404/) (2019)
- [A Topological Loss Function for Deep-Learning based Image Segmentation using Persistent Homology](https://mlsanity.com/paper/1910.01877/) (2019)
- [Topological Autoencoders](https://mlsanity.com/paper/1906.00722/) (2019)
- [Topological Graph Neural Networks](https://mlsanity.com/paper/2102.07835/) (2021)
- [Topology-Aware Segmentation Using Discrete Morse Theory](https://mlsanity.com/paper/2103.09992/) (2021)
- [Neural Persistence: A Complexity Measure for Deep Neural Networks Using Algebraic Topology](https://mlsanity.com/paper/1812.09764/) (2018)
- [Representation Topology Divergence: A Method for Comparing Neural Network Representations](https://mlsanity.com/paper/2201.00058/) (2021)
- [Learning metrics for persistence-based summaries and applications for graph classification](https://mlsanity.com/paper/1904.12189/) (2019)
- [Manifold Topology Divergence: a Framework for Comparing Data Manifolds](https://mlsanity.com/paper/2106.04024/) (2021)
- [Learning Topology-Preserving Data Representations](https://mlsanity.com/paper/2302.00136/) (2023)
- [Uncovering the Topology of Time-Varying fMRI Data using Cubical Persistence](https://mlsanity.com/paper/2006.07882/) (2020)
- [Learning Probabilistic Topological Representations Using Discrete Morse Theory](https://mlsanity.com/paper/2206.01742/) (2022)
- [Curvature Filtrations for Graph Generative Model Evaluation](https://mlsanity.com/paper/2301.12906/) (2023)
- [ToDD: Topological Compound Fingerprinting in Computer-Aided Drug Discovery](https://mlsanity.com/paper/2211.03808/) (2022)
- [Improving Self-supervised Molecular Representation Learning using Persistent Homology](https://mlsanity.com/paper/2311.17327/) (2023)

## Built on by (most cited first, 1 of 1)

- [Topological Alignment of Shared Vision-Language Embedding Space](https://mlsanity.com/paper/2510.10889/) (2025)

## Abstract (arXiv)

Efficiently adapting large Vision-Language Models (VLMs) like CLIP for few-shot learning poses challenges in balancing pre-trained knowledge retention and task-specific adaptation. Existing methods often overlook valuable structural information within the VLM's latent space. We introduce a topology-aware tuning approach integrating Representation Topology Divergence (RTD) into the Task Residual (TR) framework. By explicitly aligning the topological structures of visual and text representations using a combined RTD and Cross-Entropy loss, while freezing base VLM encoders, our method enhances few-shot performance. We optimize only lightweight Task Residual parameters, effectively leveraging topological information. Across 6 diverse benchmark datasets, our approach demonstrates significant gains, achieving an average accuracy improvement of 1-2% over relevant baseline methods in few-shot settings. This work presents an effective strategy to boost VLM few-shot capabilities by incorporating topological alignment.