Ego-VPA: Egocentric Video Understanding with Parameter-efficient Adaptation
Not on the map. Why?
Abstract (arXiv)
Video understanding typically requires fine-tuning the large backbone when adapting to new domains. In this paper, we leverage the egocentric video foundation models (Ego-VFMs) based on video-language pre-training and propose a parameter-efficient adaptation for egocentric video tasks, namely Ego-VPA. It employs a local sparse approximation for each video frame/text feature using the basis prompts, and the selected basis prompts are used to synthesize video/text prompts. Since the basis prompts are shared across frames and modalities, it models context fusion and cross-modal transfer in an efficient fashion. Experiments show that Ego-VPA excels in lightweight adaptation (with only 0.84% learnable parameters), largely improving over baselines and reaching the performance of full fine-tuning.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 42 of its 51 references that are in the map. First posted on arXiv 28 Jul 2024.
Builds on
42 of 51 references.
References · 42
- 2017Attention Is All You Needcited by 60,361
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
- 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
- 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
- 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
- 2021Learning to Prompt for Vision-Language Modelscited by 2,281
- 2022Visual Prompt Tuningcited by 1,681
- 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,620
- 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
- 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,433
- 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,245
30 more
- 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapterscited by 1,026
- 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 950
- 2021Robust fine-tuning of zero-shot modelscited by 797
- 2019VideoBERT: A Joint Model for Video and Language Representation Learningcited by 703
- 2021VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understandingcited by 602
- 2020Rescaling Egocentric Visioncited by 540
- 2019End-to-End Learning of Visual Representations from Uncurated Instructional Videoscited by 507
- 2021Less is More: ClipBERT for Video-and-Language Learning via Sparse Samplingcited by 484
- 2017AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actionscited by 449
- 2020AdapterHub: A Framework for Adapting Transformerscited by 444
- 2022Expanding Language-Image Pretrained Models for General Video Recognitioncited by 321
- 2021ActionCLIP: A New Paradigm for Video Action Recognitioncited by 320
- 2021VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Taskscited by 320
- 2021MERLOT: Multimodal Neural Script Knowledge Modelscited by 311
- 2023Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioningcited by 263
- 2020UniVL: A Unified Video and Language Pre-Training Model for Multimodal Understanding and Generationcited by 259
- 2020AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Promptscited by 249
- 2022Egocentric Video-Language Pretrainingcited by 232
- 2022Learning Video Representations from Large Language Modelscited by 205
- 2022ST-Adapter: Parameter-Efficient Image-to-Video Transfer Learningcited by 193
- 2022Frozen CLIP Models are Efficient Video Learnerscited by 182
- 2022Fine-tuned CLIP Models are Efficient Video Learnerscited by 179
- 2022Unified Vision and Language Prompt Learningcited by 148
- 2023EgoVLPv2: Egocentric Video-Language Pre-training with Fusion in the Backbonecited by 130
- 2018Charades-Ego: A Large-Scale Dataset of Paired Third and First Person Videoscited by 128
- 2022End-to-end Generative Pretraining for Multimodal Video Captioningcited by 116
- 2023Vita-CLIP: Video and text adaptive CLIP via Multimodal Promptingcited by 91
- 2022LAVENDER: Unifying Video-Language Understanding as Masked Language Modelingcited by 73
- 2023HierVL: Learning Hierarchical Video-Language Embeddingscited by 66
- 2022VoP: Text-Video Co-operative Prompt Tuning for Cross-Modal Retrievalcited by 36
Built on by
Citing papers per year: 2026: 1.
1 paper.