# Multilingual and Multimodal Topic Modelling with Pretrained Embeddings

arXiv: https://arxiv.org/abs/2211.08057 · HTML: https://mlsanity.com/paper/2211.08057/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 42 references that are in the map.
- First posted on arXiv 15 Nov 2022.

## Builds on (most cited first, 22 of 22)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Self-supervised Learning: Generative or Contrastive](https://mlsanity.com/paper/2006.08218/) (2020)
- [Contrastive Learning of Medical Visual Representations from Paired Images and Text](https://mlsanity.com/paper/2010.00747/) (2020)
- [Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation](https://mlsanity.com/paper/2004.09813/) (2020)
- [Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning](https://mlsanity.com/paper/2203.02053/) (2022)
- [A Survey on Contrastive Self-supervised Learning](https://mlsanity.com/paper/2011.00362/) (2020)
- [Multimodal datasets: misogyny, pornography, and malignant stereotypes](https://mlsanity.com/paper/2110.01963/) (2021)
- [WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learning](https://mlsanity.com/paper/2103.01913/) (2021)
- [Topic Modeling in Embedding Spaces](https://mlsanity.com/paper/1907.04907/) (2019)
- [Autoencoding Variational Inference For Topic Models](https://mlsanity.com/paper/1703.01488/) (2017)
- [Pre-training is a Hot Topic: Contextualized Document Embeddings Improve Topic Coherence](https://mlsanity.com/paper/2004.03974/) (2020)
- [Cross-lingual Contextualized Topic Models with Zero-shot Learning](https://mlsanity.com/paper/2004.07737/) (2020)
- [Contrastive Learning for Neural Topic Model](https://mlsanity.com/paper/2110.12764/) (2021)
- [Improving Neural Topic Models using Knowledge Distillation](https://mlsanity.com/paper/2010.02377/) (2020)
- [ATM:Adversarial-neural Topic Model](https://mlsanity.com/paper/1811.00265/) (2018)
- [Multilingual Molecular Representation Learning via Contrastive Pre-training](https://mlsanity.com/paper/2109.08830/) (2021)
- [Fine-tuning Encoders for Improved Monolingual and Zero-shot Polylingual Neural Topic Modeling](https://mlsanity.com/paper/2104.05064/) (2021)
- [Evaluation of Audio-Visual Alignments in Visually Grounded Speech Models](https://mlsanity.com/paper/2108.02562/) (2021)
- [Learning Multilingual Topics from Incomparable Corpus](https://mlsanity.com/paper/1806.04270/) (2018)

## Built on by (most cited first, 6 of 6)

- [Information Screening whilst Exploiting! Multimodal Relation Extraction with Feature Denoising and Multimodal Topic Modeling](https://mlsanity.com/paper/2305.11719/) (2023)
- [Neural Multimodal Topic Modeling: A Comprehensive Evaluation](https://mlsanity.com/paper/2403.17308/) (2024)
- [CEMTM: Contextual Embedding-based Multimodal Topic Modeling](https://mlsanity.com/paper/2509.11465/) (2025)
- [An Image is Worth K Topics: A Visual Structural Topic Model with Pretrained Image Embeddings](https://mlsanity.com/paper/2504.10004/) (2025)
- [Evaluating Cognitive-Behavioral Fixation via Multimodal User Viewing Patterns on Social Media](https://mlsanity.com/paper/2509.04823/) (2025)
- [Beyond Semantics: Modeling Factual and Affective Perceptual Experiences from Vision-Language Data](https://mlsanity.com/paper/2606.03345/) (2026)

## Abstract (arXiv)

This paper presents M3L-Contrast -- a novel multimodal multilingual (M3L) neural topic model for comparable data that maps texts from multiple languages and images into a shared topic space. Our model is trained jointly on texts and images and takes advantage of pretrained document and image embeddings to abstract the complexities between different languages and modalities. As a multilingual topic model, it produces aligned language-specific topics and as multimodal model, it infers textual representations of semantic concepts in images. We demonstrate that our model is competitive with a zero-shot topic model in predicting topic distributions for comparable multilingual data and significantly outperforms a zero-shot model in predicting topic distributions for comparable texts and images. We also show that our model performs almost as well on unaligned embeddings as it does on aligned embeddings.