# SSVD: Structured SVD for Parameter-Efficient Fine-Tuning and Benchmarking under Domain Shift in ASR

arXiv: https://arxiv.org/abs/2509.02830 · HTML: https://mlsanity.com/paper/2509.02830/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 33 references that are in the map.
- First posted on arXiv 2 Sep 2025.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models](https://mlsanity.com/paper/2106.10199/) (2021)
- [DoRA: Weight-Decomposed Low-Rank Adaptation](https://mlsanity.com/paper/2402.09353/) (2024)
- [ESPnet: End-to-End Speech Processing Toolkit](https://mlsanity.com/paper/1804.00015/) (2018)
- [VeRA: Vector-based Random Matrix Adaptation](https://mlsanity.com/paper/2310.11454/) (2023)
- [PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models](https://mlsanity.com/paper/2404.02948/) (2024)
- [QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2309.14717/) (2023)
- [Orthogonalizing Convolutional Layers with the Cayley Transform](https://mlsanity.com/paper/2104.07167/) (2021)
- [E-Branchformer: Branchformer with Enhanced merging for speech recognition](https://mlsanity.com/paper/2210.00077/) (2022)
- [OWSM v3.1: Better and Faster Open Whisper-Style Speech Models based on E-Branchformer](https://mlsanity.com/paper/2401.16658/) (2024)
- [SVFT: Parameter-Efficient Fine-Tuning with Singular Vectors](https://mlsanity.com/paper/2405.19597/) (2024)
- [Reproducing Whisper-Style Training Using an Open-Source Toolkit and Publicly Available Data](https://mlsanity.com/paper/2309.13876/) (2023)
- [LoRA-Whisper: Parameter-Efficient and Extensible Multilingual ASR](https://mlsanity.com/paper/2406.06619/) (2024)
- [Benchmarking Children's ASR with Supervised and Self-supervised Speech Foundation Models](https://mlsanity.com/paper/2406.10507/) (2024)
- [My Science Tutor (MyST) -- A Large Corpus of Children's Conversational Speech](https://mlsanity.com/paper/2309.13347/) (2023)
- [Kid-Whisper: Towards Bridging the Performance Gap in Automatic Speech Recognition for Children VS. Adults](https://mlsanity.com/paper/2309.07927/) (2023)
- [Sparsely Shared LoRA on Whisper for Child Speech Recognition](https://mlsanity.com/paper/2309.11756/) (2023)
- [OWLS: Scaling Laws for Multilingual Speech Recognition and Translation Models](https://mlsanity.com/paper/2502.10373/) (2025)
- [Leveraging Large Language Models for Exploiting ASR Uncertainty](https://mlsanity.com/paper/2309.04842/) (2023)
- [Towards Rehearsal-Free Multilingual ASR: A LoRA-based Case Study on Whisper](https://mlsanity.com/paper/2408.10680/) (2024)
- [Bottleneck Low-rank Transformers for Low-resource Spoken Language Understanding](https://mlsanity.com/paper/2206.14318/) (2022)

## Built on by (most cited first, 4 of 4)

- [A Language-Agnostic Hierarchical LoRA-MoE Architecture for CTC-based Multilingual ASR](https://mlsanity.com/paper/2601.00557/) (2026)
- [SSVD-O: Parameter-Efficient Fine-Tuning with Structured SVD for Speech Recognition](https://mlsanity.com/paper/2601.12600/) (2026)
- [AntiPaSTO: Self-Supervised Honesty Steering via Anti-Parallel Representations](https://mlsanity.com/paper/2601.07473/) (2026)
- [Frozen Factor or Spectral Band? Disentangling Two Choices in Low-Rank LoRA](https://mlsanity.com/paper/2610.06621/) (2026)

## Abstract (arXiv)

Parameter-efficient fine-tuning (PEFT) has emerged as a scalable solution for adapting large foundation models. While low-rank adaptation (LoRA) is widely used in speech applications, its state-of-the-art variants, e.g., VeRA, DoRA, PiSSA, and SVFT, are developed mainly for language and vision tasks, with limited validation in speech. This work presents the first comprehensive integration and benchmarking of these PEFT methods within ESPnet. We further introduce structured SVD-guided (SSVD) fine-tuning, which selectively rotates input-associated right singular vectors while keeping output-associated vectors fixed to preserve semantic mappings. This design enables robust domain adaptation with minimal trainable parameters and improved efficiency. We evaluate all methods on domain-shifted speech recognition tasks, including child speech and dialectal variation, across model scales from 0.1B to 2B. All implementations are released in ESPnet to support reproducibility and future work.