mlsanityData to 14 Oct 2026

End-to-end audio-visual learning for cochlear implant sound coding simulations in noisy environments

arXiv 2508.13576 ↗First posted 19 Aug 2025eess.AS cs.AI cs.SDcited by 0

Not on the map. Why?

Abstract (arXiv)

The cochlear implant (CI) is a successful biomedical device that enables individuals with severe-to-profound hearing loss to perceive sound through electrical stimulation, yet listening in noise remains challenging. Recent deep learning advances offer promising potential for CI sound coding by integrating visual cues. In this study, an audio-visual speech enhancement (AVSE) module is integrated with the ElectrodeNet-CS (ECS) model to form the end-to-end CI system, AVSE-ECS. Simulations show that the AVSE-ECS system with joint training achieves high objective speech intelligibility and improves the signal-to-error ratio (SER) by 7.4666 dB compared to the advanced combination encoder (ACE) strategy. These findings underscore the potential of AVSE-based CI sound coding.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 13 of its 36 references that are in the map. First posted on arXiv 19 Aug 2025.

Builds on

13 of 36 references.

References · 13

  1. 2019MediaPipe: A Framework for Building Perception Pipelinescited by 583
  2. 2020An Overview of Deep-Learning-Based Audio-Visual Speech Enhancement and Separationcited by 81
  3. 2017Audio-Visual Speech Enhancement Using Multimodal Deep Convolutional Neural Networkscited by 36
  4. 2019Audio-visual Speech Enhancement Using Conditional Variational Auto-Encoderscited by 27
  5. 2024IndicVoices-R: Unlocking a Massive Multilingual Multi-speaker Speech Corpus for Scaling Indian TTScited by 16
  6. 2018Lip-Reading Driven Deep Learning Approach for Speech Enhancementcited by 14
  7. 2020Lite Audio-Visual Speech Enhancementcited by 12
  8. 2020Improved Lite Audio-Visual Speech Enhancementcited by 12
  9. 2025Universal Speech Enhancement with Regression and Generative Mambacited by 7
  10. 2023Leveraging Self-Supervised Audio-Visual Pretrained Models to Improve Vocoded Speech Intelligibility in Cochlear Implant Simulationcited by 4
  11. 2025Bridging The Multi-Modality Gaps of Audio, Visual and Linguistic for Speech Enhancementcited by 3
  12. 2023ElectrodeNet -- A Deep Learning Based Sound Coding Strategy for Cochlear Implantscited by 2
1 more
  1. 2025NeuroAMP: A Novel End-to-end General Purpose Deep Neural Amplifier for Personalized Hearing Aidscited by 2

Built on by

Not cited yet.