mlsanityData to 7 Oct 2026

Cortex Inspired Learning to Recover Damaged Signal Modality with ReD-SOM Model

arXiv 2307.15095 ↗First posted 27 Jul 2023q-bio.NC cs.AI cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent progress in the fields of AI and cognitive sciences opens up new challenges that were previously inaccessible to study. One of such modern tasks is recovering lost data of one modality by using the data from another one. A similar effect (called the McGurk Effect) has been found in the functioning of the human brain. Observing this effect, one modality of information interferes with another, changing its perception. In this paper, we propose a way to simulate such an effect and use it to reconstruct lost data modalities by combining Variational Auto-Encoders, Self-Organizing Maps, and Hebb connections in a unified ReD-SOM (Reentering Deep Self-organizing Map) model. We are inspired by human's capability to use different zones of the brain in different modalities, in case of having a lack of information in one of the modalities. This new approach not only improves the analysis of ambiguous data but also restores the intended signal! The results obtained on the multimodal dataset demonstrate an increase of quality of the signal reconstruction. The effect is remarkable both visually and quantitatively, specifically in presence of a significant degree of signal's distortion.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 16 of its 48 references that are in the map. First posted on arXiv 27 Jul 2023.

Builds on

16 of 48 references.

References · 16

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2022Generative Adversarial Networkscited by 5,603
  6. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  7. 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 741
  8. 2018SOM-VAE: Interpretable Discrete Representation Learning on Time Seriescited by 59
  9. 2019Mixture of Inference Networks for VAE-based Audio-visual Speech Enhancementcited by 11
  10. 2020Machine Learning in Magnetic Resonance Imaging: Image Reconstructioncited by 8
  11. 2020Improving Self-Organizing Maps with Unsupervised Feature Extractioncited by 5
  12. 2022SOM-CPC: Unsupervised Contrastive Learning with Self-Organizing Maps for Structured Representations of High-Rate Time Seriescited by 4
4 more
  1. 2018Deep Neural Mapscited by 2
  2. 2020Deep Categorization with Semi-Supervised Self-Organizing Mapscited by 2
  3. 2020Brain-inspired self-organization with cellular neuromorphic computing for multimodal unsupervised learningcited by 1
  4. 2022A unified software/hardware scalable architecture for brain-inspired computing based on self-organizing neural modelscited by 1

Built on by

Not cited yet.