# CycleGAN-based Non-parallel Speech Enhancement with an Adaptive Attention-in-attention Mechanism

arXiv: https://arxiv.org/abs/2107.13143 · HTML: https://mlsanity.com/paper/2107.13143/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jul 2021.

## Built on by (most cited first, 7 of 7)

- [MetricGAN-U: Unsupervised speech enhancement/ dereverberation based only on noisy/ reverberated speech](https://mlsanity.com/paper/2110.05866/) (2021)
- [Dual-branch Attention-In-Attention Transformer for single-channel speech enhancement](https://mlsanity.com/paper/2110.06467/) (2021)
- [Exploiting Time-Frequency Conformers for Music Audio Enhancement](https://mlsanity.com/paper/2308.12599/) (2023)
- [A Hybrid Model for Weakly-Supervised Speech Dereverberation](https://mlsanity.com/paper/2502.06839/) (2025)
- [A Training and Inference Strategy Using Noisy and Enhanced Speech as Target for Speech Enhancement without Clean Speech](https://mlsanity.com/paper/2210.15368/) (2022)
- [Unsupervised speech enhancement with deep dynamical generative speech and noise models](https://mlsanity.com/paper/2306.07820/) (2023)
- [Capacity-Net-Based RIS Precoding Design without Channel Estimation for mmWave MIMO System](https://mlsanity.com/paper/2509.25660/) (2025)

## Abstract (arXiv)

Non-parallel training is a difficult but essential task for DNN-based speech enhancement methods, for the lack of adequate noisy and paired clean speech corpus in many real scenarios. In this paper, we propose a novel adaptive attention-in-attention CycleGAN (AIA-CycleGAN) for non-parallel speech enhancement. In previous CycleGAN-based non-parallel speech enhancement methods, the limited mapping ability of the generator may cause performance degradation and insufficient feature learning. To alleviate this degradation, we propose an integration of adaptive time-frequency attention (ATFA) and adaptive hierarchical attention (AHA) to form an attention-in-attention (AIA) module for more flexible feature learning during the mapping procedure. More specifically, ATFA can capture the long-range temporal-spectral contextual information for more effective feature representations, while AHA can flexibly aggregate different AFTA's intermediate output feature maps by adaptive attention weights depending on the global context. Numerous experimental results demonstrate that the proposed approach achieves consistently more superior performance over previous GAN-based and CycleGAN-based methods in non-parallel training. Moreover, experiments in parallel training verify that the proposed AIA-CycleGAN also outperforms most advanced GAN-based and Non-GAN based speech enhancement approaches, especially in maintaining speech integrity and reducing speech distortion.