# EEG-informed attended speaker extraction from recorded speech mixtures with application in neuro-steered hearing prostheses

arXiv: https://arxiv.org/abs/1602.05702 · HTML: https://mlsanity.com/paper/1602.05702/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Feb 2016.

## Built on by (most cited first, 6 of 6)

- [Egocentric Auditory Attention Localization in Conversations](https://mlsanity.com/paper/2303.16024/) (2023)
- [A DenseNet-based method for decoding auditory spatial attention with EEG](https://mlsanity.com/paper/2309.07690/) (2023)
- [Listen, Chat, and Remix: Text-Guided Soundscape Remixing for Enhanced Auditory Experience](https://mlsanity.com/paper/2402.03710/) (2024)
- [DGSD: Dynamical Graph Self-Distillation for EEG-Based Auditory Spatial Attention Detection](https://mlsanity.com/paper/2309.07147/) (2023)
- [S2M-Former: Spiking Symmetric Mixing Branchformer for Brain Auditory Attention Detection](https://mlsanity.com/paper/2508.05164/) (2025)
- [TFGA-Net: Temporal-Frequency Graph Attention Network for Brain-Controlled Speaker Extraction](https://mlsanity.com/paper/2510.12275/) (2025)

## Abstract (arXiv)

OBJECTIVE: We aim to extract and denoise the attended speaker in a noisy, two-speaker acoustic scenario, relying on microphone array recordings from a binaural hearing aid, which are complemented with electroencephalography (EEG) recordings to infer the speaker of interest. METHODS: In this study, we propose a modular processing flow that first extracts the two speech envelopes from the microphone recordings, then selects the attended speech envelope based on the EEG, and finally uses this envelope to inform a multi-channel speech separation and denoising algorithm. RESULTS: Strong suppression of interfering (unattended) speech and background noise is achieved, while the attended speech is preserved. Furthermore, EEG-based auditory attention detection (AAD) is shown to be robust to the use of noisy speech signals. CONCLUSIONS: Our results show that AAD-based speaker extraction from microphone array recordings is feasible and robust, even in noisy acoustic environments, and without access to the clean speech signals to perform EEG-based AAD. SIGNIFICANCE: Current research on AAD always assumes the availability of the clean speech signals, which limits the applicability in real settings. We have extended this research to detect the attended speaker even when only microphone recordings with noisy speech mixtures are available. This is an enabling ingredient for new brain-computer interfaces and effective filtering schemes in neuro-steered hearing prostheses. Here, we provide a first proof of concept for EEG-informed attended speaker extraction and denoising.