mlsanityData to 7 Oct 2026

Contrastive Learning for Neural Topic Model

arXiv 2110.12764 ↗First posted 25 Oct 2021cs.CLcited by 48

Not on the map. Why?

Abstract (arXiv)

Recent empirical studies show that adversarial topic models (ATM) can successfully capture semantic patterns of the document by differentiating a document with another dissimilar sample. However, utilizing that discriminative-generative architecture has two important drawbacks: (1) the architecture does not relate similar documents, which has the same document-word distribution of salient words; (2) it restricts the ability to integrate external information, such as sentiments of the document, which has been shown to benefit the training of neural topic model. To address those issues, we revisit the adversarial topic architecture in the viewpoint of mathematical analysis, propose a novel approach to re-formulate discriminative goal as an optimization problem, and design a novel sampling method which facilitates the integration of external variables. The reformulation encourages the model to incorporate the relations among similar samples and enforces the constraint on the similarity among dissimilar ones; while the sampling method, which is based on the internal input and reconstructed output, helps inform the model of salient words contributing to the main topic. Experimental results show that our framework outperforms other state-of-the-art neural topic models in three common benchmark datasets that belong to various domains, vocabulary sizes, and document lengths in terms of topic coherence.

Key facts. Cited by 48 papers in the map (data through 7 Oct 2026). Builds on 42 of its 56 references that are in the map. First posted on arXiv 25 Oct 2021.

Builds on

42 of 56 references.

References · 42

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  3. 2016Pointer Sentinel Mixture Modelscited by 2,992
  4. 2020Supervised Contrastive Learningcited by 2,803
  5. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
  6. 2019Contrastive Multiview Codingcited by 1,297
  7. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
  8. 2020Graph Contrastive Learning with Augmentationscited by 1,203
  9. 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 918
  10. 2020What Makes for Good Views for Contrastive Learning?cited by 843
  11. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 733
  12. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 672
30 more
  1. 2020Contrastive Multi-View Representation Learning on Graphscited by 651
  2. 2020Contrastive Learning with Hard Negative Samplescited by 619
  3. 2019InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximizationcited by 441
  4. 2020Debiased Contrastive Learningcited by 354
  5. 2020Contrastive learning of global and local features for medical image segmentation with limited annotationscited by 309
  6. 2021DetCo: Unsupervised Contrastive Learning for Object Detectioncited by 205
  7. 2019Graph Matching Networks for Learning the Similarity of Graph Structured Objectscited by 202
  8. 2018An efficient framework for learning sentence representationscited by 197
  9. 2017Autoencoding Variational Inference For Topic Modelscited by 168
  10. 2021FSCE: Few-Shot Object Detection via Contrastive Proposal Encodingcited by 168
  11. 2013A Survey on Multi-view Learningcited by 152
  12. 2020Adversarial Self-Supervised Contrastive Learningcited by 138
  13. 2015Neural Variational Inference for Text Processingcited by 111
  14. 2020Contrastive Learning for Label-Efficient Semantic Segmentationcited by 93
  15. 2020Contrastive Learning with Adversarial Examplescited by 91
  16. 2017Discovering Discrete Latent Topics with Neural Variational Inferencecited by 79
  17. 2021Dual Contrastive Learning for Unsupervised Image-to-Image Translationcited by 65
  18. 2019Topic Modeling with Wasserstein Autoencoderscited by 53
  19. 2017Neural Models for Documents with Metadatacited by 46
  20. 2021Universal Weakly Supervised Segmentation by Pixel-to-Segment Contrastive Learningcited by 43
  21. 2018Unsupervised Hard Example Mining from Videos for Improved Object Detectioncited by 30
  22. 2020Neural Topic Modeling with Bidirectional Adversarial Trainingcited by 29
  23. 2018Coherence-Aware Neural Topic Modelingcited by 26
  24. 2020Improving Neural Topic Models using Knowledge Distillationcited by 25
  25. 2018ATM:Adversarial-neural Topic Modelcited by 23
  26. 2021Enriching and Controlling Global Semantics for Text Summarizationcited by 20
  27. 2019Capturing Greater Context for Question Generationcited by 19
  28. 2020Neural Topic Modeling with Cycle-Consistent Adversarial Trainingcited by 12
  29. 2019Open Event Extraction from Online Text using a Generative Adversarial Networkcited by 10
  30. 2021How Pandemic Spread in News: Text Analysis Using Topic Modelcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 5; 2023: 12; 2024: 19; 2025: 9; 2026: 2.

48 papers.

Sort:

By area:

  1. 2023Effective Neural Topic Modeling with Embedding Clustering Regularizationcited by 46
  2. 2022Mitigating Data Sparsity for Short Text Topic Modeling by Topic-Semantic Contrastive Learningcited by 38
  3. 2024Video-Language Understanding: A Survey from Model Architecture, Model Training, and Data Perspectivescited by 37
  4. 2024Heterogeneous Contrastive Learning for Foundation Models and Beyondcited by 27
  5. 2021Improving Neural Cross-Lingual Summarization via Employing Optimal Transport Distance for Knowledge Distillationcited by 22
  6. 2023Towards the TopMost: A Topic Modeling System Toolkitcited by 21
  7. 2022A Simple Contrastive Learning Objective for Alleviating Neural Text Degenerationcited by 18
  8. 2024On the Affinity, Rationality, and Diversity of Hierarchical Topic Modelingcited by 18
  9. 2024A Survey on Neural Topic Models: Methods, Applications, and Challengescited by 18
  10. 2024Modeling Dynamic Topics in Chain-Free Fashion by Evolution-Tracking Contrastive Learning and Unassociated Word Exclusioncited by 16
  11. 2022Adaptive Contrastive Learning on Multimodal Transformer for Review Helpfulness Predictionscited by 13
  12. 2024KDMCSE: Knowledge Distillation Multimodal Sentence Embeddings with Adaptive Angular margin Contrastive Learningcited by 13
36 more
  1. 2023Improving Multimodal Sentiment Analysis: Supervised Angular Margin-based Contrastive Learning for Enhanced Fusion Representationcited by 12
  2. 2024Topic Modeling as Multi-Objective Contrastive Optimizationcited by 12
  3. 2023DemaFormer: Damped Exponential Moving Average Transformer with Energy-Based Modeling for Temporal Language Groundingcited by 11
  4. 2024LLM Reading Tea Leaves: Automatically Evaluating Topic Models with Large Language Modelscited by 11
  5. 2024Neural Topic Modeling with Large Language Models in the Loopcited by 10
  6. 2024NeuroMax: Enhancing Neural Topic Modeling via Maximizing Mutual Information and Group Topic Regularizationcited by 9
  7. 2024Motion-aware Contrastive Learning for Temporal Panoptic Scene Graph Generationcited by 9
  8. 2022Exploiting Contrastive Learning and Numerical Evidence for Confusing Legal Judgment Predictioncited by 8
  9. 2024Encoding and Controlling Global Semantics for Long-form Video Question Answeringcited by 8
  10. 2024MAMA: Meta-optimized Angular Margin Contrastive Framework for Video-Language Representation Learningcited by 8
  11. 2025TI-JEPA: An Innovative Energy-based Joint Embedding Strategy for Text-Image Multimodal Systemscited by 8
  12. 2025Dynamic User-controllable Privacy-preserving Few-shot Sensing Frameworkcited by 8
  13. 2023Gradient-Boosted Decision Tree for Listwise Context Model in Multimodal Review Helpfulness Predictioncited by 7
  14. 2022Multilingual and Multimodal Topic Modelling with Pretrained Embeddingscited by 6
  15. 2023Expand BERT Representation with Visual Information via Grounded Language Learning with Multimodal Partial Alignmentcited by 5
  16. 2024Enhancing Short-Text Topic Modeling with LLM-Driven Context Expansion and Prefix-Tuned VAEscited by 5
  17. 2024Curriculum Demonstration Selection for In-Context Learningcited by 5
  18. 2024Multi-Scale Contrastive Learning for Video Temporal Groundingcited by 5
  19. 2023Improving Contextualized Topic Models with Negative Samplingcited by 4
  20. 2024GloCOM: A Short Text Neural Topic Model via Global Clustering Contextcited by 4
  21. 2025CutPaste&Find: Efficient Multimodal Hallucination Detector with Visual-aid Knowledge Basecited by 4
  22. 2025Enhancing Multimodal Entity Linking with Jaccard Distance-based Conditional Contrastive Learning and Contextual Visual Augmentationcited by 3
  23. 2025XTRA: Cross-Lingual Topic Modeling with Topic and Representation Alignmentscited by 3
  24. 2025Sparse Autoencoders are Topic Modelscited by 3
  25. 2023Reinforcement Learning for Topic Modelscited by 1
  26. 2023Graph Contrastive Topic Modelcited by 1
  27. 2023Towards Generalising Neural Topical Representationscited by 1
  28. 2024Topic-aware Causal Intervention for Counterfactual Detectioncited by 1
  29. 2024Enhancing Topic Interpretability for Neural Topic Modeling through Topic-wise Contrastive Learningcited by 1
  30. 2026LLM-XTM: Enhancing Cross-Lingual Topic Models with Large Language Modelscited by 1
  31. 2023What are Public Concerns about ChatGPT? A Novel Self-Supervised Neural Topic Model Tells Youcited by 0
  32. 2023Let the Pretrained Language Models "Imagine" for Short Texts Topic Modelingcited by 0
  33. 2025Self-Supervised Learning for Neural Topic Models with Variance-Invariance-Covariance Regularizationcited by 0
  34. 2025Evaluating Negative Sampling Approaches for Neural Topic Modelscited by 0
  35. 2025Understanding Cross-Domain Adaptation in Low-Resource Topic Modelingcited by 0
  36. 2026A Neural Topic Method Using a Large-Language-Model-in-the-Loop for Business Researchcited by 0