# Multi-Metric Optimization using Generative Adversarial Networks for Near-End Speech Intelligibility Enhancement

arXiv: https://arxiv.org/abs/2104.08499 · HTML: https://mlsanity.com/paper/2104.08499/

## Key facts

- Cited by 1 papers in the map (data through 19 Oct 2026).
- First posted on arXiv 17 Apr 2021.

## Built on by (most cited first, 1 of 1)

- [Joint Noise Reduction and Listening Enhancement for Full-End Speech Enhancement](https://mlsanity.com/paper/2203.11500/) (2022)

## Abstract (arXiv)

The intelligibility of speech severely degrades in the presence of environmental noise and reverberation. In this paper, we propose a novel deep learning based system for modifying the speech signal to increase its intelligibility under the equal-power constraint, i.e., signal power before and after modification must be the same. To achieve this, we use generative adversarial networks (GANs) to obtain time-frequency dependent amplification factors, which are then applied to the input raw speech to reallocate the speech energy. Instead of optimizing only a single, simple metric, we train a deep neural network (DNN) model to simultaneously optimize multiple advanced speech metrics, including both intelligibility- and quality-related ones, which results in notable improvements in performance and robustness. Our system can not only work in non-realtime mode for offline audio playback but also support practical real-time speech applications. Experimental results using both objective measurements and subjective listening tests indicate that the proposed system significantly outperforms state-ofthe-art baseline systems under various noisy and reverberant listening conditions.