mlsanityData to 7 Oct 2026

ST-Adapter: Parameter-Efficient Image-to-Video Transfer Learning

arXiv 2206.13559 ↗First posted 27 Jun 2022cs.CVcited by 192

Not on the map. Why?

Cited by 21 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Capitalizing on large pre-trained models for various downstream tasks of interest have recently emerged with promising performance. Due to the ever-growing model size, the standard full fine-tuning based task adaptation strategy becomes prohibitively costly in terms of model training and storage. This has led to a new research direction in parameter-efficient transfer learning. However, existing attempts typically focus on downstream tasks from the same modality (e.g., image understanding) of the pre-trained model. This creates a limit because in some specific modalities, (e.g., video understanding) such a strong pre-trained model with sufficient knowledge is less or not available. In this work, we investigate such a novel cross-modality transfer learning setting, namely parameter-efficient image-to-video transfer learning. To solve this problem, we propose a new Spatio-Temporal Adapter (ST-Adapter) for parameter-efficient fine-tuning per video task. With a built-in spatio-temporal reasoning capability in a compact design, ST-Adapter enables a pre-trained image model without temporal knowledge to reason about dynamic video content at a small (~8%) per-task parameter cost, requiring approximately 20 times fewer updated parameters compared to previous work. Extensive experiments on video action recognition tasks show that our ST-Adapter can match or even outperform the strong full fine-tuning strategy and state-of-the-art video models, whilst enjoying the advantage of parameter efficiency. The code and model are available at https://github.com/linziyi96/st-adapter

Key facts. Cited by 192 papers in the map (data through 7 Oct 2026). Builds on 82 of its 112 references that are in the map. First posted on arXiv 27 Jun 2022.

Builds on

82 of 112 references.

References · 82

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  6. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  7. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  8. 2016Layer Normalizationcited by 5,579
  9. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  10. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  11. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  12. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
38 more
  1. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  2. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  3. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  4. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  5. 2017Non-local Neural Networkscited by 2,290
  6. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  7. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  8. 2017The Kinetics Human Action Video Datasetcited by 1,894
  9. 2018SlowFast Networks for Video Recognitioncited by 1,862
  10. 2022Visual Prompt Tuningcited by 1,674
  11. 2021ViViT: A Video Vision Transformercited by 1,610
  12. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,609
  13. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,427
  14. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,305
  15. 2019Contrastive Multiview Codingcited by 1,297
  16. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,265
  17. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,234
  18. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,139
  19. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,092
  20. 2021Video Swin Transformercited by 1,090
  21. 2021Multiscale Vision Transformerscited by 896
  22. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 863
  23. 2018TSM: Temporal Shift Module for Efficient Video Understandingcited by 852
  24. 2020AdapterFusion: Non-Destructive Task Composition for Transfer Learningcited by 780
  25. 2019VideoBERT: A Joint Model for Video and Language Representation Learningcited by 702
  26. 2021P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Taskscited by 686
  27. 2020X3D: Expanding Architectures for Efficient Video Recognitioncited by 635
  28. 2021VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understandingcited by 596
  29. 2021Masked Feature Prediction for Self-Supervised Visual Pre-Trainingcited by 573
  30. 2017Learning multiple visual domains with residual adapterscited by 553
  31. 2020Rescaling Egocentric Visioncited by 529
  32. 2017Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classificationcited by 521
  33. 2021MViTv2: Improved Multiscale Vision Transformers for Classification and Detectioncited by 516
  34. 2020AdapterHub: A Framework for Adapting Transformerscited by 441
  35. 2021VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Textcited by 427
  36. 2021Learning How to Ask: Querying LMs with Mixtures of Soft Promptscited by 376
  37. 2018A Short Note about Kinetics-600cited by 331
  38. 2021VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Taskscited by 319

50 most cited of 82 All 82 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 10; 2023: 55; 2024: 53; 2025: 51; 2026: 23.

192 papers.

Sort:

By area:

  1. 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 641
  2. 2023Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessmentcited by 227
  3. 2023DreamVideo: Composing Your Dream Videos with Customized Subject and Motioncited by 186
  4. 2023What Makes Good Examples for Visual In-Context Learning?cited by 150
  5. 2023SimDA: Simple Diffusion Adapter for Efficient Video Generationcited by 101
  6. 2022VindLU: A Recipe for Effective Video-and-Language Pretrainingcited by 87
  7. 2024A Survey of Resource-efficient LLM and Multimodal Foundation Modelscited by 84
  8. 2023Modular Deep Learningcited by 82
  9. 2023MA-SAM: Modality-agnostic SAM Adaptation for 3D Medical Image Segmentationcited by 82
  10. 2023Foundation Model for Endoscopy Video Analysis via Large-scale Self-supervised Pre-traincited by 75
  11. 2024Holistic Autonomous Driving Understanding by Bird's-Eye-View Injected Multi-Modal Large Modelscited by 75
  12. 2022Vision Transformers are Parameter-Efficient Audio-Visual Learnerscited by 72
38 more
  1. 20233DSAM-adapter: Holistic adaptation of SAM from 2D to 3D for promptable tumor segmentationcited by 70
  2. 2024CricaVPR: Cross-image Correlation-aware Representation Learning for Visual Place Recognitioncited by 61
  3. 2023Revisiting Temporal Modeling for CLIP-based Image-to-Video Knowledge Transferringcited by 57
  4. 2023Open-VCLIP: Transforming CLIP to an Open-vocabulary Video Model via Interpolated Weight Optimizationcited by 54
  5. 2024Towards Seamless Adaptation of Pre-trained Models for Visual Place Recognitioncited by 49
  6. 2022Bidirectional Cross-Modal Knowledge Exploration for Video Recognition with Pre-trained Vision-Language Modelscited by 48
  7. 2023HiLM-D: Enhancing MLLMs with Multi-Scale High-Resolution Details for Autonomous Drivingcited by 48
  8. 2022PIVOT: Prompting for Video Continual Learningcited by 45
  9. 2023Implicit Temporal Modeling with Learnable Alignment for Video Recognitioncited by 44
  10. 2025Parameter-Efficient Fine-Tuning for Foundation Modelscited by 43
  11. 2023BT-Adapter: Video Conversation is Feasible Without Video Instruction Tuningcited by 41
  12. 2023UniAdapter: Unified Parameter-Efficient Transfer Learning for Cross-modal Modelingcited by 40
  13. 2023Dual-path Adaptation from Image to Video Transformerscited by 40
  14. 2023From Static to Dynamic: Adapting Landmark-Aware Image Models for Facial Expression Recognition in Videoscited by 39
  15. 2022Cross-Modal Adapter for Vision-Language Retrievalcited by 35
  16. 2024FROSTER: Frozen CLIP Is A Strong Teacher for Open-Vocabulary Action Recognitioncited by 35
  17. 2023Visual Tuningcited by 34
  18. 2023A Simple Recipe for Contrastively Pre-training Video-First Encoders Beyond 16 Framescited by 34
  19. 2024R2-Tuning: Efficient Image-to-Video Transfer Learning for Video Temporal Groundingcited by 34
  20. 2023eP-ALM: Efficient Perceptual Augmentation of Language Modelscited by 33
  21. 2025Exploring the Evolution of Physics Cognition in Video Generation: A Surveycited by 31
  22. 2023VicTR: Video-conditioned Text Representations for Activity Recognitioncited by 28
  23. 2024Time-, Memory- and Parameter-Efficient Visual Adaptationcited by 28
  24. 2024AWT: Transferring Vision-Language Models via Augmentation, Weighting, and Transportationcited by 27
  25. 2022Towards a Unified View on Visual Parameter-Efficient Transfer Learningcited by 25
  26. 2024Dynamic Tuning Towards Parameter and Inference Efficiency for ViT Adaptationcited by 25
  27. 2023Parameter-efficient is not sufficient: Exploring Parameter, Memory, and Time Efficient Adapter Tuning for Dense Predictionscited by 24
  28. 2023Parameter-Efficient Transfer Learning for Remote Sensing Image-Text Retrievalcited by 24
  29. 2023ViLA: Efficient Video-Language Alignment for Video Question Answeringcited by 23
  30. 2025Video-STAR: Reinforcing Open-Vocabulary Action Recognition with Toolscited by 23
  31. 2023VLN-PETL: Parameter-Efficient Transfer Learning for Vision-and-Language Navigationcited by 22
  32. 2024Mitigating the Human-Robot Domain Discrepancy in Visual Pre-training for Robotic Manipulationcited by 22
  33. 2023VL-PET: Vision-and-Language Parameter-Efficient Tuning via Granularity Controlcited by 21
  34. 2023Building an Open-Vocabulary Video CLIP Model with Better Architectures, Optimization and Datacited by 21
  35. 2024Parameter-Efficient Fine-Tuning for Pre-Trained Vision Models: A Survey and Benchmarkcited by 21
  36. 2023End-to-End Spatio-Temporal Action Localisation with Video Transformerscited by 20
  37. 2023SCT: A Simple Baseline for Parameter-Efficient Fine-Tuning via Salient Channelscited by 20
  38. 2025Modeling Fine-Grained Hand-Object Dynamics for Egocentric Video Representation Learningcited by 20

50 most cited of 192 All 192 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023AIM: Adapting Image Models for Efficient Video Action Recognition
  2. 2024Cross-Modal Adapter: Parameter-Efficient Transfer Learning Approach for Vision-Language Models
  3. 2023READ: Recurrent Adapter with Partial Video-Language Alignment for Parameter-Efficient Transfer Learning in Low-Resource Video-Language Modeling
  4. 2025RMAdapter: Reconstruction-based Multi-Modal Adapter for Vision-Language Models
  5. 2024AID: Adapting Image2Video Diffusion Models for Instruction-guided Video Prediction
  6. 2023Disentangling Spatial and Temporal Learning for Efficient Image-to-Video Transfer Learning
  7. 2026Edit2Interp: Adapting Image Foundation Models from Spatial Editing to Video Frame Interpolation with Few-Shot Learning
  8. 2025Video-LMM Post-Training: A Deep Dive into Video Reasoning with Large Multimodal Models