# Improved Lite Audio-Visual Speech Enhancement

arXiv: https://arxiv.org/abs/2008.13222 · HTML: https://mlsanity.com/paper/2008.13222/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Aug 2020.

## Built on by (most cited first, 12 of 12)

- [MeLFusion: Synthesizing Music from Image and Language Cues using Diffusion Models](https://mlsanity.com/paper/2406.04673/) (2024)
- [Diffusion-based Unsupervised Audio-visual Speech Enhancement](https://mlsanity.com/paper/2410.05301/) (2024)
- [A Preliminary Study of a Two-Stage Paradigm for Preserving Speaker Identity in Dysarthric Voice Conversion](https://mlsanity.com/paper/2106.01415/) (2021)
- [Real-Time Audio-Visual Speech Enhancement Using Pre-trained Visual Representations](https://mlsanity.com/paper/2507.21448/) (2025)
- [LSTMSE-Net: Long Short Term Speech Enhancement Network for Audio-visual Speech Enhancement](https://mlsanity.com/paper/2409.02266/) (2024)
- [Bridging The Multi-Modality Gaps of Audio, Visual and Linguistic for Speech Enhancement](https://mlsanity.com/paper/2501.13375/) (2025)
- [EMGSE: Acoustic/EMG Fusion for Multimodal Speech Enhancement](https://mlsanity.com/paper/2202.06507/) (2022)
- [Deep denoising autoencoder-based non-invasive blood flow detection for arteriovenous fistula](https://mlsanity.com/paper/2306.06865/) (2023)
- [Unveiling the frontiers of deep learning: innovations shaping diverse domains](https://mlsanity.com/paper/2309.02712/) (2023)
- [End-to-end audio-visual learning for cochlear implant sound coding simulations in noisy environments](https://mlsanity.com/paper/2508.13576/) (2025)
- [Real-Time System for Audio-Visual Target Speech Enhancement](https://mlsanity.com/paper/2509.20741/) (2025)
- [Visual-Informed Speech Enhancement Using Attention-Based Beamforming](https://mlsanity.com/paper/2603.05270/) (2026)

## Abstract (arXiv)

Numerous studies have investigated the effectiveness of audio-visual multimodal learning for speech enhancement (AVSE) tasks, seeking a solution that uses visual data as auxiliary and complementary input to reduce the noise of noisy speech signals. Recently, we proposed a lite audio-visual speech enhancement (LAVSE) algorithm for a car-driving scenario. Compared to conventional AVSE systems, LAVSE requires less online computation and to some extent solves the user privacy problem on facial data. In this study, we extend LAVSE to improve its ability to address three practical issues often encountered in implementing AVSE systems, namely, the additional cost of processing visual data, audio-visual asynchronization, and low-quality visual data. The proposed system is termed improved LAVSE (iLAVSE), which uses a convolutional recurrent neural network architecture as the core AVSE model. We evaluate iLAVSE on the Taiwan Mandarin speech with video dataset. Experimental results confirm that compared to conventional AVSE systems, iLAVSE can effectively overcome the aforementioned three practical issues and can improve enhancement performance. The results also confirm that iLAVSE is suitable for real-world scenarios, where high-quality audio-visual sensors may not always be available.