mlsanityData to 7 Oct 2026

Hellinger Multimodal Variational Autoencoders

arXiv 2601.06572 ↗First posted 10 Jan 2026cs.LG cs.AIcited by 2

On the map: Mixture-of-experts → Multimodal MoE models

Abstract (arXiv)

Multimodal variational autoencoders (VAEs) are widely used for weakly supervised generative learning with multiple modalities. Predominant methods aggregate unimodal inference distributions using either a product of experts (PoE), a mixture of experts (MoE), or their combinations to approximate the joint posterior. In this work, we revisit multimodal inference through the lens of probabilistic opinion pooling, an optimization-based approach. We start from H"older pooling with α=0.5, which corresponds to the unique symmetric member of the α-divergence family, and derive a moment-matching approximation, termed Hellinger. We then leverage such an approximation to propose HELVAE, a multimodal VAE that avoids sub-sampling, yielding an efficient yet effective model that: (i) learns more expressive latent representations as additional modalities are observed; and (ii) empirically achieves better trade-offs between generative coherence and quality, outperforming state-of-the-art multimodal VAE models.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Multimodal MoE models, one of 12 directions in Mixture-of-experts. Builds on 37 of its 74 references that are in the map. First posted on arXiv 10 Jan 2026.

Builds on

37 of 74 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Multimodal Variational Autoencoder: a Barycentric View29 Dec
  2. 2023Learning multi-modal generative models with permutation-invariant encoders and tighter variational objectives1 Sep
  3. 2019Variational Mixture-of-Experts Autoencoders for Multi-Modal Deep Generative Models8 Nov

Other references · 34

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  4. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  5. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  6. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  7. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 773
  8. 2024Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Designcited by 294
  9. 2021Cross-Modal Contrastive Learning for Text-to-Image Generationcited by 259
  10. 2018Multimodal Generative Models for Scalable Weakly-Supervised Learningcited by 200
  11. 2019Controllable Text-to-Image Generationcited by 187
  12. 2016Joint Multimodal Learning with Deep Generative Modelscited by 87
22 more
  1. 2021Generalized Multimodal ELBOcited by 86
  2. 2020Multimodal Generative Learning Utilizing Jensen-Shannon-Divergencecited by 48
  3. 2017Generative Models of Visually Grounded Imaginationcited by 44
  4. 2023Subtractive Mixture Models via Squaring: Representation and Learningcited by 39
  5. 2015Black-box α-divergence Minimizationcited by 37
  6. 2021On the Limitations of Multimodal VAEscited by 37
  7. 2022Mitigating Modality Collapse in Multimodal VAEs via Impartial Optimizationcited by 32
  8. 2022Fusion of Probability Density Functionscited by 22
  9. 2024Efficient LLM Comparative Assessment: a Product of Experts Framework for Pairwise Comparisonscited by 22
  10. 2024Sum of Squares Circuitscited by 21
  11. 2025Diffuse Everything: Multimodal Diffusion Models on Arbitrary State Spacescited by 19
  12. 2021Learning Multimodal VAEs through Mutual Supervisioncited by 18
  13. 2024Unity by Diversity: Improved Representation Learning in Multimodal VAEscited by 13
  14. 2023Multi-modal Latent Diffusioncited by 9
  15. 2024Diffusion Models For Multi-Modal Generative Modelingcited by 9
  16. 2024Learning Multimodal Latent Generative Models with Energy-Based Priorcited by 7
  17. 2023Score-Based Multimodal Autoencodercited by 6
  18. 2025Product of Experts for Visual Generationcited by 6
  19. 2025How to Square Tensor Networks and Circuits Without Squaring Themcited by 5
  20. 2025Aggregation of Dependent Expert Distributions in Multimodal Variational Autoencoderscited by 3
  21. 2024A Markov Random Field Multi-Modal Variational AutoEncodercited by 2
  22. 2025Bridging the inference gap in Mutimodal Variational Autoencoderscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026MUNI: Multimodal Unified Latent Diffusion for Coherent Any-to-Any Generationcited by 1
  2. 2026H"older++: Improving the Quality-Coherence Trade-off in Multimodal VAEscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021Multimodal Variational Autoencoders for Semi-Supervised Learning: In Defense of Product-of-Experts
  2. 2025CoVAE: Consistency Training of Variational Autoencoders
  3. 2022Generalizing Multimodal Variational Methods to Sets
  4. 2025A Frequentist Statistical Introduction to Variational Inference, Autoencoders, and Diffusion Models
  5. 2022Mixture-of-experts VAEs can disregard variation in surjective multimodal data
  6. 2021Diffusion Priors In Variational Autoencoders
  7. 2024Multimodal ELBO with Diffusion Decoders
  8. 2026MUNITE: Unified Multimodal Latent Inference for Any-to-Any Multimodal Generation