# MambAdapter: Lightweight Mamba-Based Adapters for Parameter-Efficient Transfer Learning in Speech and Audio

arXiv: https://arxiv.org/abs/2606.15638 · HTML: https://mlsanity.com/paper/2606.15638/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Speech and audio SSMs, one of 10 directions in State-space sequence models (S4, Mamba).
- Builds on 30 of its 34 references that are in the map.
- First posted on arXiv 14 Jun 2026.

## Placed in

- State-space sequence models (S4, Mamba) / [Speech and audio SSMs](https://mlsanity.com/ssm/speech-and-audio-ssms/)

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models](https://mlsanity.com/paper/2106.10199/) (2021)
- [Towards a Unified View of Parameter-Efficient Transfer Learning](https://mlsanity.com/paper/2110.04366/) (2021)
- [AdapterFusion: Non-Destructive Task Composition for Transfer Learning](https://mlsanity.com/paper/2005.00247/) (2020)
- [AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition](https://mlsanity.com/paper/2205.13535/) (2022)
- [AST: Audio Spectrogram Transformer](https://mlsanity.com/paper/2104.01778/) (2021)
- [Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworks](https://mlsanity.com/paper/2106.04489/) (2021)
- [Speech Model Pre-training for End-to-End Spoken Language Understanding](https://mlsanity.com/paper/1904.03670/) (2019)
- [Convolutional Bypasses Are Better Vision Transformer Adapters](https://mlsanity.com/paper/2207.07039/) (2022)
- [Mamba in Speech: Towards an Alternative to Self-Attention](https://mlsanity.com/paper/2405.12609/) (2024)
- [Open-Source Conversational AI with SpeechBrain 1.0](https://mlsanity.com/paper/2407.00463/) (2024)
- [Dual-path Mamba: Short and Long-term Bidirectional Selective Structured State Space Models for Speech Separation](https://mlsanity.com/paper/2403.18257/) (2024)
- [Efficient Adapter Transfer of Self-Supervised Speech Models for Automatic Speech Recognition](https://mlsanity.com/paper/2202.03218/) (2022)
- [An Adapter Based Pre-Training for Efficient and Scalable Self-Supervised Speech Representation Learning](https://mlsanity.com/paper/2107.13530/) (2021)
- [Audio Mamba: Selective State Spaces for Self-Supervised Audio Representations](https://mlsanity.com/paper/2406.02178/) (2024)
- [CL-MASR: A Continual Learning Benchmark for Multilingual ASR](https://mlsanity.com/paper/2310.16931/) (2023)
- [Parameter Efficient Transfer Learning for Various Speech Processing Tasks](https://mlsanity.com/paper/2212.02780/) (2022)
- [Parameter-Efficient Transfer Learning of Audio Spectrogram Transformers](https://mlsanity.com/paper/2312.03694/) (2023)
- [Mamba for Streaming ASR Combined with Unimodal Aggregation](https://mlsanity.com/paper/2410.00070/) (2024)
- [Efficient Fine-tuning of Audio Spectrogram Transformers via Soft Mixture of Adapters](https://mlsanity.com/paper/2402.00828/) (2024)
- [The Curious Case of In-Training Compression of State Space Models](https://mlsanity.com/paper/2510.02823/) (2025)

## Abstract (arXiv)

Fine-tuning Transformer-based foundation models has become the dominant strategy for domain adaptation in audio and speech processing. To reduce the computational and memory costs of this process, parameter-efficient transfer learning (PETL) methods have been widely explored. Meanwhile, Mamba, a recent state-space model, has emerged as a promising alternative to Transformers for sequence modeling. In this work, we present MambAdapter, a parameter-efficient transfer learning approach that integrates Mamba into low-rank bottleneck adapters. Our design combines parameter sharing across adapters with the injection of a lightweight Mamba module, enabling more effective modeling of audio features. We demonstrate that MambAdapter matches or outperforms strong PETL baselines on four audio classification tasks and five speech recognition languages, even when operating under reduced parameter budgets.