Neural Topic Modeling with Deep Mutual Information Estimation
Not on the map. Why?
Abstract (arXiv)
The emerging neural topic models make topic modeling more easily adaptable and extendable in unsupervised text mining. However, the existing neural topic models is difficult to retain representative information of the documents within the learnt topic representation. In this paper, we propose a neural topic model which incorporates deep mutual information estimation, i.e., Neural Topic Modeling with Deep Mutual Information Estimation(NTM-DMIE). NTM-DMIE is a neural network method for topic learning which maximizes the mutual information between the input documents and their latent topic representation. To learn robust topic representation, we incorporate the discriminator to discriminate negative examples and positive examples via adversarial learning. Moreover, we use both global and local mutual information to preserve the rich information of the input documents in the topic representation. We evaluate NTM-DMIE on several metrics, including accuracy of text clustering, with topic representation, topic uniqueness and topic coherence. Compared to the existing methods, the experimental results show that NTM-DMIE can outperform in all the metrics on the four datasets.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 26 of its 42 references that are in the map. First posted on arXiv 12 Mar 2022.
Builds on
26 of 42 references.
References · 26
- 2022Generative Adversarial Networkscited by 5,603
- 2013Auto-Encoding Variational Bayescited by 3,980
- 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
- 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
- 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 733
- 2018MINE: Mutual Information Neural Estimationcited by 680
- 2016f-GAN: Training Generative Neural Samplers using Variational Divergence Minimizationcited by 504
- 2019Topic Modeling in Embedding Spacescited by 224
- 2017Autoencoding Variational Inference For Topic Modelscited by 168
- 2015Neural Variational Inference for Text Processingcited by 111
- 2017Discovering Discrete Latent Topics with Neural Variational Inferencecited by 79
- 2019Deep Spectral Clustering using Dual Autoencoder Networkcited by 71
14 more
- 2019Topic Modeling with Wasserstein Autoencoderscited by 53
- 2019Learning Disentangled Representations via Mutual Information Estimationcited by 52
- 2020Neural Topic Model via Optimal Transportcited by 50
- 2017Neural Models for Documents with Metadatacited by 46
- 2020Neural Topic Modeling with Bidirectional Adversarial Trainingcited by 29
- 2018Coherence-Aware Neural Topic Modelingcited by 26
- 2018ATM:Adversarial-neural Topic Modelcited by 23
- 2020A Batch Normalized Inference Network Keeps the KL Vanishing Awaycited by 20
- 2018Document Informed Neural Autoregressive Topic Models with Distributional Priorcited by 11
- 2020TAN-NTM: Topic Attention Networks for Neural Topic Modelingcited by 10
- 2021Neural Attention-Aware Hierarchical Topic Modelcited by 6
- 2021Multi-source Neural Topic Modeling in Multi-view Embedding Spacescited by 4
- 2021TopNet: Learning from Neural Topic Model to Generate Long Storiescited by 2
- 2020Matching Text with Deep Mutual Information Estimationcited by 1
Built on by
Citing papers per year: 2023: 1.
1 paper.