# TUNet: A Block-online Bandwidth Extension Model based on Transformers and Self-supervised Pretraining

arXiv: https://arxiv.org/abs/2110.13492 · HTML: https://mlsanity.com/paper/2110.13492/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 23 references that are in the map.
- First posted on arXiv 26 Oct 2021.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Rethinking Attention with Performers](https://mlsanity.com/paper/2009.14794/) (2020)
- [SDR - half-baked or well done?](https://mlsanity.com/paper/1811.02508/) (2018)
- [Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram](https://mlsanity.com/paper/1910.11480/) (2019)
- [DNSMOS: A Non-Intrusive Perceptual Objective Speech Quality metric to evaluate Noise Suppressors](https://mlsanity.com/paper/2010.15258/) (2020)
- [Learning long-range spatial dependencies with horizontal gated-recurrent units](https://mlsanity.com/paper/1805.08315/) (2018)
- [Effectiveness of self-supervised pre-training for speech recognition](https://mlsanity.com/paper/1911.03912/) (2019)
- [NU-Wave: A Diffusion Probabilistic Model for Neural Audio Upsampling](https://mlsanity.com/paper/2104.02321/) (2021)
- [Unsupervised Pre-training of Bidirectional Speech Encoders via Masked Reconstruction](https://mlsanity.com/paper/2001.10603/) (2020)
- [Temporal FiLM: Capturing Long-Range Sequence Dependencies with Feature-Wise Modulations](https://mlsanity.com/paper/1909.06628/) (2019)
- [Audio Super Resolution using Neural Networks](https://mlsanity.com/paper/1708.00853/) (2017)
- [WSRGlow: A Glow-based Waveform Generative Model for Audio Super-Resolution](https://mlsanity.com/paper/2106.08507/) (2021)
- [On Filter Generalization for Music Bandwidth Extension Using Deep Neural Networks](https://mlsanity.com/paper/2011.07274/) (2020)
- [Is the Skip Connection Provable to Reform the Neural Network Loss Landscape?](https://mlsanity.com/paper/2006.05939/) (2020)

## Built on by (most cited first, 8 of 8)

- [A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AI](https://mlsanity.com/paper/2303.13336/) (2023)
- [NU-Wave 2: A General Neural Audio Upsampling Model for Various Sampling Rates](https://mlsanity.com/paper/2206.08545/) (2022)
- [Analysing Diffusion-based Generative Approaches versus Discriminative Approaches for Speech Restoration](https://mlsanity.com/paper/2211.02397/) (2022)
- [Audio Super-Resolution with Latent Bridge Models](https://mlsanity.com/paper/2509.17609/) (2025)
- [Nonparallel High-Quality Audio Super Resolution with Domain Adaptation and Resampling CycleGANs](https://mlsanity.com/paper/2210.15887/) (2022)
- [Speech Bandwidth Expansion Via High Fidelity Generative Adversarial Networks](https://mlsanity.com/paper/2407.18571/) (2024)
- [A Dual-Branch Parallel Network for Speech Enhancement and Restoration](https://mlsanity.com/paper/2409.08702/) (2024)
- [P2Flow: Phoneme-aware Progressive Flow Matching for Extreme Speech Super-Resolution](https://mlsanity.com/paper/2609.24138/) (2026)

## Abstract (arXiv)

We introduce a block-online variant of the temporal feature-wise linear modulation (TFiLM) model to achieve bandwidth extension. The proposed architecture simplifies the UNet backbone of the TFiLM to reduce inference time and employs an efficient transformer at the bottleneck to alleviate performance degradation. We also utilize self-supervised pretraining and data augmentation to enhance the quality of bandwidth extended signals and reduce the sensitivity with respect to downsampling methods. Experiment results on the VCTK dataset show that the proposed method outperforms several recent baselines in both intrusive and non-intrusive metrics. Pretraining and filter augmentation also help stabilize and enhance the overall performance.