mlsanityData to 7 Oct 2026

Multimodal Generative Learning Utilizing Jensen-Shannon-Divergence

arXiv 2006.08242 ↗First posted 15 Jun 2020cs.LG stat.MLcited by 48

Not on the map. Why?

Abstract (arXiv)

Learning from different data types is a long-standing goal in machine learning research, as multiple information sources co-occur when describing natural phenomena. However, existing generative models that approximate a multimodal ELBO rely on difficult or inefficient training schemes to learn a joint distribution and the dependencies between modalities. In this work, we propose a novel, efficient objective function that utilizes the Jensen-Shannon divergence for multiple distributions. It simultaneously approximates the unimodal and joint multimodal posteriors directly via a dynamic prior. In addition, we theoretically prove that the new multimodal JS-divergence (mmJSD) objective optimizes an ELBO. In extensive experiments, we demonstrate the advantage of the proposed mmJSD model compared to previous work in unsupervised, generative learning tasks.

Key facts. Cited by 48 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 4; 2023: 8; 2024: 11; 2025: 10; 2026: 7.

48 papers.

Sort:

By area:

  1. 2022MultiMAE: Multi-modal Multi-task Masked Autoencoderscited by 287
  2. 2021Generalized Multimodal ELBOcited by 86
  3. 2022A survey of multimodal deep generative modelscited by 47
  4. 2021On the Limitations of Multimodal VAEscited by 37
  5. 2023Multimodal Variational Auto-encoder based Audio-Visual Segmentationcited by 37
  6. 2022Mitigating Modality Collapse in Multimodal VAEs via Impartial Optimizationcited by 32
  7. 2021Learning Multimodal VAEs through Mutual Supervisioncited by 18
  8. 2024Unity by Diversity: Improved Representation Learning in Multimodal VAEscited by 13
  9. 2024Modeling Inter-Intra Heterogeneity for Graph Federated Learningcited by 12
  10. 2023Multi-modal Latent Diffusioncited by 9
  11. 2024Learning Multimodal Latent Generative Models with Energy-Based Priorcited by 7
  12. 2025Deep Incomplete Multi-view Learning via Cyclic Permutation of VAEscited by 7
36 more
  1. 2021Fast Multi-Step Critiquing for VAE-based Recommender Systemscited by 6
  2. 2023A multimodal dynamical variational autoencoder for audiovisual speech representation learningcited by 6
  3. 2024Text-to-Image Cross-Modal Generation: A Systematic Reviewcited by 5
  4. 2021Discriminative Multimodal Learning via Conditional Priors in Generative Modelscited by 4
  5. 2023Learning multi-modal generative models with permutation-invariant encoders and tighter variational objectivescited by 4
  6. 2021Composable Generative Modelscited by 3
  7. 2023Pneumonia Detection in Chest X-Ray Images : Handling Class Imbalancecited by 3
  8. 2023Improving Normative Modeling for Multi-modal Neuroimaging Data using mixture-of-product-of-experts variational autoencoderscited by 3
  9. 2026Inference-Time Dynamic Modality Selection for Incomplete Multimodal Classificationcited by 3
  10. 2021Multimodal Adversarially Learned Inference with Factorized Discriminatorscited by 2
  11. 2022Benchmarking Multimodal Variational Autoencoders: CdSprites+ Dataset and Toolkitcited by 2
  12. 2024A Markov Random Field Multi-Modal Variational AutoEncodercited by 2
  13. 2024Multimodal ELBO with Diffusion Decoderscited by 2
  14. 2024From Concrete to Abstract: A Multimodal Generative Approach to Abstract Concept Learningcited by 2
  15. 2025Multi-Component VAE with Gaussian Markov Random Fieldcited by 2
  16. 2026Hellinger Multimodal Variational Autoencoderscited by 2
  17. 2023TC-VAE: Uncovering Out-of-Distribution Data Generative Factorscited by 1
  18. 2023Improving Multimodal Joint Variational Autoencoders through Normalizing Flows and Correlation Analysiscited by 1
  19. 2024A tutorial on multi-view autoencoders using the multi-view-AE librarycited by 1
  20. 2024Learning Multimodal Latent Space with EBM Prior and MCMC Inferencecited by 1
  21. 2025ShaLa: Multimodal Shared Latent Space Modellingcited by 1
  22. 2025Surgeons Are Indian Males and Speech Therapists Are White Females: Auditing Biases in Vision-Language Models for Healthcare Professionalscited by 1
  23. 2025DiA-gnostic VLVAE: Disentangled Alignment-Constrained Vision Language Variational AutoEncoder for Robust Radiology Reporting with Missing Modalitiescited by 1
  24. 2025Towards Generalizable Deepfake Detection via Forgery-aware Audio-Visual Adaptation: A Variational Bayesian Approachcited by 1
  25. 2026Instruction-Driven 3D Facial Expression Generation and Transitioncited by 1
  26. 2026T-DuMpRa: Teacher-guided Dual-path Multi-prototype Retrieval Augmented framework for fine-grained medical image classificationcited by 1
  27. 2021Recommending Burgers based on Pizza Preferences: Addressing Data Sparsity with a Product of Expertscited by 0
  28. 2024Enhancing Unimodal Latent Representations in Multimodal VAEs through Iterative Amortized Inferencecited by 0
  29. 2024Correlating Variational Autoencoders Natively For Multi-View Imputationcited by 0
  30. 2025Enhanced uncertainty quantification variational autoencoders for the solution of Bayesian inverse problemscited by 0
  31. 2025FedRecon: Missing Modality Reconstruction in Heterogeneous Distributed Environmentscited by 0
  32. 2025Deep Generative Methods and Tire Architecture Designcited by 0
  33. 2025Two tales for a geometric Jensen--Shannon divergencecited by 0
  34. 2026One-Shot Federated Clustering of Non-Independent Completely Distributed Datacited by 0
  35. 2026Multimodal normative modeling in Alzheimers Disease with introspective variational autoencoderscited by 0
  36. 2026Characterizing the Predictive Impact of Modalities with Supervised Latent-Variable Modelingcited by 0