# TB-AVA: Text as a Semantic Bridge for Audio-Visual Parameter Efficient Finetuning

arXiv: https://arxiv.org/abs/2605.11572 · HTML: https://mlsanity.com/paper/2605.11572/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 33 references that are in the map.
- First posted on arXiv 12 May 2026.

## Builds on (most cited first, 27 of 27)

- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [Conditional Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2203.05557/) (2022)
- [Swin Transformer V2: Scaling Up Capacity and Resolution](https://mlsanity.com/paper/2111.09883/) (2021)
- [AdapterFusion: Non-Destructive Task Composition for Transfer Learning](https://mlsanity.com/paper/2005.00247/) (2020)
- [VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs](https://mlsanity.com/paper/2406.07476/) (2024)
- [Qwen2.5-Omni Technical Report](https://mlsanity.com/paper/2503.20215/) (2025)
- [BEATs: Audio Pre-Training with Acoustic Tokenizers](https://mlsanity.com/paper/2212.09058/) (2022)
- [Attention Bottlenecks for Multimodal Fusion](https://mlsanity.com/paper/2107.00135/) (2021)
- [AudioCLIP: Extending CLIP to Image, Text and Audio](https://mlsanity.com/paper/2106.13043/) (2021)
- [HTS-AT: A Hierarchical Token-Semantic Audio Transformer for Sound Classification and Detection](https://mlsanity.com/paper/2202.00874/) (2022)
- [Audio-Visual Segmentation](https://mlsanity.com/paper/2207.05042/) (2022)
- [Unified Multisensory Perception: Weakly-Supervised Audio-Visual Video Parsing](https://mlsanity.com/paper/2007.10558/) (2020)
- [A Closer Look at Weakly-Supervised Audio-Visual Source Localization](https://mlsanity.com/paper/2209.09634/) (2022)
- [Vision Transformers are Parameter-Efficient Audio-Visual Learners](https://mlsanity.com/paper/2212.07983/) (2022)
- [Cross-modal Prompts: Adapting Large Pre-trained Models for Audio-Visual Downstream Tasks](https://mlsanity.com/paper/2311.05152/) (2023)
- [Joint-Modal Label Denoising for Weakly-Supervised Audio-Visual Video Parsing](https://mlsanity.com/paper/2204.11573/) (2022)
- [Towards Open-Vocabulary Audio-Visual Event Localization](https://mlsanity.com/paper/2411.11278/) (2024)
- [T-VSL: Text-Guided Visual Sound Source Localization in Mixtures](https://mlsanity.com/paper/2404.01751/) (2024)
- [Improving Audio-Visual Video Parsing with Pseudo Visual Labels](https://mlsanity.com/paper/2303.02344/) (2023)
- [Modality-Independent Teachers Meet Weakly-Supervised Audio-Visual Event Parser](https://mlsanity.com/paper/2305.17343/) (2023)
- [Mettle: Meta-Token Learning for Memory-Efficient Audio-Visual Adaptation](https://mlsanity.com/paper/2506.23271/) (2025)
- [TAViS: Text-bridged Audio-Visual Segmentation with Foundation Models](https://mlsanity.com/paper/2506.11436/) (2025)
- [MoLT: Mixture of Layer-Wise Tokens for Efficient Audio-Visual Learning](https://mlsanity.com/paper/2512.00115/) (2025)

## Abstract (arXiv)

Audio-visual understanding requires effective alignment between heterogeneous modalities, yet cross-modal correspondence remains challenging when temporally aligned audio and visual signals lack clear semantic correspondence. We propose to use text as a semantic anchor for audio-visual representation learning. To this end, we introduce a parameter-efficient adaptation framework built on frozen audio and visual encoders, centered on Text-Bridged Audio-Visual Adapter (TB-AVA), which enables text-mediated interaction between audio and visual streams. At the core of TB-AVA, Gated Semantic Modulation (GSM) selectively modulates feature channels based on text-inferred semantic relevance. We evaluate the proposed approach on multiple benchmarks, including AVE, AVS, and AVVP, where the proposed framework achieves state-of-the-art performance, demonstrating text as an effective semantic anchor for parameter-efficient fine-tuning (PEFT) in audio-visual learning.