# RedApt: An Adaptor for wav2vec 2 Encoding  Faster and Smaller Speech Translation without Quality Compromise

arXiv: https://arxiv.org/abs/2210.08475 · HTML: https://mlsanity.com/paper/2210.08475/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 28 references that are in the map.
- First posted on arXiv 16 Oct 2022.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [Multilingual Denoising Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2001.08210/) (2020)
- [SUPERB: Speech processing Universal PERformance Benchmark](https://mlsanity.com/paper/2105.01051/) (2021)
- [Multilingual Translation with Extensible Multilingual Pretraining and Finetuning](https://mlsanity.com/paper/2008.00401/) (2020)
- [fairseq S2T: Fast Speech-to-Text Modeling with fairseq](https://mlsanity.com/paper/2010.05171/) (2020)
- [Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing](https://mlsanity.com/paper/2006.03236/) (2020)
- [DistilHuBERT: Speech Representation Learning by Layer-wise Distillation of Hidden-unit BERT](https://mlsanity.com/paper/2110.01900/) (2021)
- [Performance-Efficiency Trade-offs in Unsupervised Pre-training for Speech Recognition](https://mlsanity.com/paper/2109.06870/) (2021)
- [PARP: Prune, Adjust and Re-Prune for Self-Supervised Speech Recognition](https://mlsanity.com/paper/2106.05933/) (2021)
- [End-to-end Speech Translation via Cross-modal Progressive Training](https://mlsanity.com/paper/2104.10380/) (2021)
- [CTC-based Compression for Direct Speech Translation](https://mlsanity.com/paper/2102.01578/) (2021)
- [Adaptive Feature Selection for End-to-End Speech Translation](https://mlsanity.com/paper/2010.08518/) (2020)
- [Shrinking Bigfoot: Reducing wav2vec 2.0 footprint](https://mlsanity.com/paper/2103.15760/) (2021)
- [End-to-End Speech Translation with Pre-trained Models and Adapters: UPC at IWSLT 2021](https://mlsanity.com/paper/2105.04512/) (2021)
- [M-Adapter: Modality Adaptation for End-to-End Speech-to-Text Translation](https://mlsanity.com/paper/2207.00952/) (2022)
- [Phone Features Improve Speech Translation](https://mlsanity.com/paper/2005.13681/) (2020)

## Built on by (most cited first, 2 of 2)

- [Pushing the Limits of Zero-shot End-to-End Speech Translation](https://mlsanity.com/paper/2402.10422/) (2024)
- [How do Hyenas deal with Human Speech? Speech Recognition and Translation with ConfHyena](https://mlsanity.com/paper/2402.13208/) (2024)

## Abstract (arXiv)

Pre-trained speech Transformers in speech translation (ST) have facilitated state-of-the-art (SotA) results; yet, using such encoders is computationally expensive. To improve this, we present a novel Reducer Adaptor block, RedApt, that could be seamlessly integrated within any Transformer-based speech encoding architecture. Integrating the pretrained wav2vec 2 speech encoder with RedAptbrings 41% speedup, 33% memory reduction with 24% fewer FLOPs at inference. To our positive surprise, our ST model with RedApt outperforms the SotA architecture by an average of 0.68 BLEU score on 8 language pairs from Must-C.