# Neural network-based virtual microphone estimation with virtual microphone and beamformer-level multi-task loss

arXiv: https://arxiv.org/abs/2311.11595 · HTML: https://mlsanity.com/paper/2311.11595/

## Key facts

- Cited by 1 papers in the map (data through 19 Oct 2026).
- First posted on arXiv 20 Nov 2023.

## Built on by (most cited first, 1 of 1)

- [Spatial-Magnifier: Spatial upsampling for multichannel speech enhancement](https://mlsanity.com/paper/2605.04749/) (2026)

## Abstract (arXiv)

Array processing performance depends on the number of microphones available. Virtual microphone estimation (VME) has been proposed to increase the number of microphone signals artificially. Neural network-based VME (NN-VME) trains an NN with a VM-level loss to predict a signal at a microphone location that is available during training but not at inference. However, this training objective may not be optimal for a specific array processing back-end, such as beamforming. An alternative approach is to use a training objective considering the array-processing back-end, such as a loss on the beamformer output. This approach may generate signals optimal for beamforming but not physically grounded. To combine the advantages of both approaches, this paper proposes a multi-task loss for NN-VME that combines both VM-level and beamformer-level losses. We evaluate the proposed multi-task NN-VME on multi-talker underdetermined conditions and show that it achieves a 33.1 % relative WER improvement compared to using only real microphones and 10.8 % compared to using a prior NN-VME approach.