# Complex spectrogram enhancement by convolutional neural network with multi-metrics learning

arXiv: https://arxiv.org/abs/1704.08504 · HTML: https://mlsanity.com/paper/1704.08504/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Apr 2017.

## Built on by (most cited first, 17 of 17)

- [Speech Enhancement and Dereverberation with Diffusion-based Generative Models](https://mlsanity.com/paper/2208.05830/) (2022)
- [TSTNN: Two-stage Transformer based Neural Network for Speech Enhancement in the Time Domain](https://mlsanity.com/paper/2103.09963/) (2021)
- [Complex-valued Convolutional Neural Networks for Enhanced Radar Signal Denoising and Interference Mitigation](https://mlsanity.com/paper/2105.00929/) (2021)
- [Speech Enhancement Using Continuous Embeddings of Neural Audio Codec](https://mlsanity.com/paper/2502.16240/) (2025)
- [Resource-efficient Deep Neural Networks for Automotive Radar Interference Mitigation](https://mlsanity.com/paper/2201.10360/) (2022)
- [Rethinking complex-valued deep neural networks for monaural speech enhancement](https://mlsanity.com/paper/2301.04320/) (2023)
- [DPSNN: Spiking Neural Network for Low-Latency Streaming Speech Enhancement](https://mlsanity.com/paper/2408.07388/) (2024)
- [A Composite Predictive-Generative Approach to Monaural Universal Speech Enhancement](https://mlsanity.com/paper/2505.24576/) (2025)
- [Deep neural network techniques for monaural speech enhancement: state of the art analysis](https://mlsanity.com/paper/2212.00369/) (2022)
- [Bridging The Multi-Modality Gaps of Audio, Visual and Linguistic for Speech Enhancement](https://mlsanity.com/paper/2501.13375/) (2025)
- [A Survey of Deep Learning for Complex Speech Spectrograms](https://mlsanity.com/paper/2505.08694/) (2025)
- [Pre-training Feature Guided Diffusion Model for Speech Enhancement](https://mlsanity.com/paper/2406.07646/) (2024)
- [Leveraging Heteroscedastic Uncertainty in Learning Complex Spectral Mapping for Single-channel Speech Enhancement](https://mlsanity.com/paper/2211.08624/) (2022)
- [DeepExtractor: Time-domain reconstruction of signals and glitches in gravitational wave data with deep learning](https://mlsanity.com/paper/2501.18423/) (2025)
- [Audio Spectral Enhancement: Leveraging Autoencoders for Low Latency Reconstruction of Long, Lossy Audio Sequences](https://mlsanity.com/paper/2108.03703/) (2021)
- [SEOFP-NET: Compression and Acceleration of Deep Neural Networks for Speech Enhancement Using Sign-Exponent-Only Floating-Points](https://mlsanity.com/paper/2111.04436/) (2021)
- [Speech Separation based on Contrastive Learning and Deep Modularization](https://mlsanity.com/paper/2305.10652/) (2023)

## Abstract (arXiv)

This paper aims to address two issues existing in the current speech enhancement methods: 1) the difficulty of phase estimations; 2) a single objective function cannot consider multiple metrics simultaneously. To solve the first problem, we propose a novel convolutional neural network (CNN) model for complex spectrogram enhancement, namely estimating clean real and imaginary (RI) spectrograms from noisy ones. The reconstructed RI spectrograms are directly used to synthesize enhanced speech waveforms. In addition, since log-power spectrogram (LPS) can be represented as a function of RI spectrograms, its reconstruction is also considered as another target. Thus a unified objective function, which combines these two targets (reconstruction of RI spectrograms and LPS), is equivalent to simultaneously optimizing two commonly used objective metrics: segmental signal-to-noise ratio (SSNR) and logspectral distortion (LSD). Therefore, the learning process is called multi-metrics learning (MML). Experimental results confirm the effectiveness of the proposed CNN with RI spectrograms and MML in terms of improved standardized evaluation metrics on a speech enhancement task.