mlsanityData to 7 Oct 2026

TEST-V: TEst-time Support-set Tuning for Zero-shot Video Classification

arXiv 2502.00426 ↗First posted 1 Feb 2025cs.CVcited by 2

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Recently, adapting Vision Language Models (VLMs) to zero-shot visual classification by tuning class embedding with a few prompts (Test-time Prompt Tuning, TPT) or replacing class names with generated visual samples (support-set) has shown promising results. However, TPT cannot avoid the semantic gap between modalities while the support-set cannot be tuned. To this end, we draw on each other's strengths and propose a novel framework namely TEst-time Support-set Tuning for zero-shot Video Classification (TEST-V). It first dilates the support-set with multiple prompts (Multi-prompting Support-set Dilation, MSD) and then erodes the support-set via learnable weights to mine key cues dynamically (Temporal-aware Support-set Erosion, TSE). Specifically, i) MSD expands the support samples for each class based on multiple prompts enquired from LLMs to enrich the diversity of the support-set. ii) TSE tunes the support-set with factorized learnable weights according to the temporal prediction consistency in a self-supervised manner to dig pivotal supporting cues for each class. TEST-V achieves state-of-the-art results across four benchmarks and has good interpretability for the support-set dilation and erosion.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 34 of its 51 references that are in the map. First posted on arXiv 1 Feb 2025.

Builds on

34 of 51 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Dual Memory Networks: A Versatile Adaptation Approach for Vision-Language Models26 Mar
  2. 2022SuS-X: Training-Free Name-Only Transfer of Vision-Language Models28 Nov
  3. 2022Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models15 Sep
  4. 2022Conditional Prompt Learning for Vision-Language Models10 Mar
  5. 2021Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modeling6 Nov
  6. 2021Learning to Prompt for Vision-Language Models2 Sep

Other references · 28

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  5. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  6. 2018SlowFast Networks for Video Recognitioncited by 1,862
  7. 2018TSM: Temporal Shift Module for Efficient Video Understandingcited by 852
  8. 2021MEMO: Test Time Robustness via Adaptation and Augmentationcited by 376
  9. 2023LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Modelscited by 349
  10. 2018A Short Note about Kinetics-600cited by 331
  11. 2022Expanding Language-Image Pretrained Models for General Video Recognitioncited by 320
  12. 2021ActionCLIP: A New Paradigm for Video Action Recognitioncited by 317
16 more
  1. 2021Prompting Visual-Language Models for Efficient Video Understandingcited by 309
  2. 2023Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generationcited by 300
  3. 2017"Zero-Shot" Super-Resolution using Deep Internal Learningcited by 285
  4. 2022What does a platypus look like? Generating customized prompts for zero-shot image classificationcited by 263
  5. 2022Fine-tuned CLIP Models are Efficient Video Learnerscited by 178
  6. 2023Prompt, Generate, then Cache: Cascade of Foundation Models makes Strong Few-shot Learnerscited by 152
  7. 2023Vita-CLIP: Video and text adaptive CLIP via Multimodal Promptingcited by 90
  8. 2022SePiCo: Semantic-Guided Pixel Contrast for Domain Adaptive Semantic Segmentationcited by 87
  9. 2021Elaborative Rehearsal for Zero-shot Action Recognitioncited by 76
  10. 2023A Recipe for Scaling up Text-to-Video Generation with Text-free Videoscited by 51
  11. 2023Hierarchical Spatio-temporal Decoupling for Text-to-Video Generationcited by 50
  12. 2022Bidirectional Cross-Modal Knowledge Exploration for Video Recognition with Pre-trained Vision-Language Modelscited by 48
  13. 2020Temporal Interlacing Networkcited by 45
  14. 2023MAtch, eXpand and Improve: Unsupervised Finetuning for Zero-Shot Action Recognition with Language Knowledgecited by 43
  15. 2023VicTR: Video-conditioned Text Representations for Activity Recognitioncited by 28
  16. 2017Zero-Shot Activity Recognition with Verb Attribute Inductioncited by 15

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Prompt-based Adaptation in Large-scale Vision Models: A Surveycited by 21
  2. 2025Adapting Vision-Language Models Without Labels: A Comprehensive Surveycited by 6

Similar, not cited

Close in wording, no citation either way

  1. 2024BaFTA: Backprop-Free Test-Time Adaptation For Zero-Shot Vision-Language Models
  2. 2024Just Shift It: Test-Time Prototype Shifting for Zero-Shot Generalization with Vision-Language Models
  3. 2025No Labels Needed: Zero-Shot Image Classification with Collaborative Self-Learning
  4. 2023Videoprompter: an ensemble of foundational models for zero-shot video understanding
  5. 2024Adapting Vision-Language Models to Open Classes via Test-Time Prompt Tuning
  6. 2022Revisiting Classifier: Transferring Vision-Language Models for Video Recognition
  7. 2025Prompting without Panic: Attribute-aware, Zero-shot, Test-Time Calibration
  8. 2024Efficient Test-Time Prompt Tuning for Vision-Language Models