# Lite Audio-Visual Speech Enhancement

arXiv: https://arxiv.org/abs/2005.11769 · HTML: https://mlsanity.com/paper/2005.11769/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 May 2020.

## Built on by (most cited first, 12 of 12)

- [VoViT: Low Latency Graph-based Audio-Visual Voice Separation Transformer](https://mlsanity.com/paper/2203.04099/) (2022)
- [EMA2S: An End-to-End Multimodal Articulatory-to-Speech System](https://mlsanity.com/paper/2102.03786/) (2021)
- [Audio-Visual Speech Separation in Noisy Environments with a Lightweight Iterative Model](https://mlsanity.com/paper/2306.00160/) (2023)
- [Real-Time Audio-Visual Speech Enhancement Using Pre-trained Visual Representations](https://mlsanity.com/paper/2507.21448/) (2025)
- [Large-scale unsupervised audio pre-training for video-to-speech synthesis](https://mlsanity.com/paper/2306.15464/) (2023)
- [Bridging The Multi-Modality Gaps of Audio, Visual and Linguistic for Speech Enhancement](https://mlsanity.com/paper/2501.13375/) (2025)
- [EPG2S: Speech Generation and Speech Enhancement based on Electropalatography and Audio Signals using Multimodal Learning](https://mlsanity.com/paper/2206.07860/) (2022)
- [AV-CrossNet: an Audiovisual Complex Spectral Mapping Network for Speech Separation By Leveraging Narrow- and Cross-Band Modeling](https://mlsanity.com/paper/2406.11619/) (2024)
- [Efficient Audiovisual Speech Processing via MUTUD: Multimodal Training and Unimodal Deployment](https://mlsanity.com/paper/2501.18157/) (2025)
- [Cooperative Dual Attention for Audio-Visual Speech Enhancement with Facial Cues](https://mlsanity.com/paper/2311.14275/) (2023)
- [End-to-end audio-visual learning for cochlear implant sound coding simulations in noisy environments](https://mlsanity.com/paper/2508.13576/) (2025)
- [Visual-Informed Speech Enhancement Using Attention-Based Beamforming](https://mlsanity.com/paper/2603.05270/) (2026)

## Abstract (arXiv)

Previous studies have confirmed the effectiveness of incorporating visual information into speech enhancement (SE) systems. Despite improved denoising performance, two problems may be encountered when implementing an audio-visual SE (AVSE) system: (1) additional processing costs are incurred to incorporate visual input and (2) the use of face or lip images may cause privacy problems. In this study, we propose a Lite AVSE (LAVSE) system to address these problems. The system includes two visual data compression techniques and removes the visual feature extraction network from the training model, yielding better online computation efficiency. Our experimental results indicate that the proposed LAVSE system can provide notably better performance than an audio-only SE system with a similar number of model parameters. In addition, the experimental results confirm the effectiveness of the two techniques for visual data compression.