# Scenario-Aware Audio-Visual TF-GridNet for Target Speech Extraction

arXiv: https://arxiv.org/abs/2310.19644 · HTML: https://mlsanity.com/paper/2310.19644/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Oct 2023.

## Built on by (most cited first, 5 of 5)

- [Plug-and-Play Co-Occurring Face Attention for Robust Audio-Visual Speaker Extraction](https://mlsanity.com/paper/2505.20635/) (2025)
- [AV-CrossNet: an Audiovisual Complex Spectral Mapping Network for Speech Separation By Leveraging Narrow- and Cross-Band Modeling](https://mlsanity.com/paper/2406.11619/) (2024)
- [Audio-Visual Speech Enhancement In Complex Scenarios With Separation And Dereverberation Joint Modeling](https://mlsanity.com/paper/2510.26825/) (2025)
- [From Speech to Interaction: Analyzing Multimodal Systems in Cocktail-Party Scenarios](https://mlsanity.com/paper/2608.08510/) (2026)
- [AVTR-1: Open Stack for Real-Time Interactive Avatars](https://mlsanity.com/paper/2609.22913/) (2026)

## Abstract (arXiv)

Target speech extraction aims to extract, based on a given conditioning cue, a target speech signal that is corrupted by interfering sources, such as noise or competing speakers. Building upon the achievements of the state-of-the-art (SOTA) time-frequency speaker separation model TF-GridNet, we propose AV-GridNet, a visual-grounded variant that incorporates the face recording of a target speaker as a conditioning factor during the extraction process. Recognizing the inherent dissimilarities between speech and noise signals as interfering sources, we also propose SAV-GridNet, a scenario-aware model that identifies the type of interfering scenario first and then applies a dedicated expert model trained specifically for that scenario. Our proposed model achieves SOTA results on the second COG-MHEAR Audio-Visual Speech Enhancement Challenge, outperforming other models by a significant margin, objectively and in a listening test. We also perform an extensive analysis of the results under the two scenarios.