mlsanityData to 7 Oct 2026

DeTiME: Diffusion-Enhanced Topic Modeling using Encoder-decoder based LLM

arXiv 2310.15296 ↗First posted 23 Oct 2023cs.CL cs.AIcited by 16

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

In the burgeoning field of natural language processing (NLP), Neural Topic Models (NTMs) , Large Language Models (LLMs) and Diffusion model have emerged as areas of significant research interest. Despite this, NTMs primarily utilize contextual embeddings from LLMs, which are not optimal for clustering or capable for topic based text generation. NTMs have never been combined with diffusion model for text generation. Our study addresses these gaps by introducing a novel framework named Diffusion-Enhanced Topic Modeling using Encoder-Decoder-based LLMs (DeTiME). DeTiME leverages Encoder-Decoder-based LLMs to produce highly clusterable embeddings that could generate topics that exhibit both superior clusterability and enhanced semantic coherence compared to existing methods. Additionally, by exploiting the power of diffusion model, our framework also provides the capability to do topic based text generation. This dual functionality allows users to efficiently produce highly clustered topics and topic based text generation simultaneously. DeTiME's potential extends to generating clustered embeddings as well. Notably, our proposed framework(both encoder-decoder based LLM and diffusion model) proves to be efficient to train and exhibits high adaptability to other LLMs and diffusion model, demonstrating its potential for a wide array of applications.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 52 of its 108 references that are in the map. First posted on arXiv 23 Oct 2023.

Builds on

52 of 108 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023The Curse of Recursion: Training on Generated Data Makes Models Forget27 May
  2. 2022Text Generation with Diffusion Language Models: A Pre-training Approach with Continuous Paragraph Denoise22 Dec
  3. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models17 Oct

Other references · 49

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2020Language Models are Few-Shot Learnerscited by 32,199
  6. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  7. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  8. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  9. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  10. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  11. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  12. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
37 more
  1. 2022Generative Adversarial Networkscited by 5,603
  2. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  3. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  4. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  5. 2020Longformer: The Long-Document Transformercited by 3,362
  6. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  7. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  8. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 973
  9. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
  10. 2022BERTopic: Neural topic modeling with a class-based TF-IDF procedurecited by 779
  11. 2018Universal Sentence Encodercited by 485
  12. 2021Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Modelscited by 462
  13. 2021Large Dual Encoders Are Generalizable Retrieverscited by 454
  14. 2017Supervised Learning of Universal Sentence Representations from Natural Language Inference Datacited by 428
  15. 2018Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Networkcited by 415
  16. 2020Evaluation of Text Generation: A Surveycited by 275
  17. 2019Topic Modeling in Embedding Spacescited by 224
  18. 2019Leveraging Pre-trained Checkpoints for Sequence Generation Taskscited by 182
  19. 2017Autoencoding Variational Inference For Topic Modelscited by 168
  20. 2020Pre-training is a Hot Topic: Contextualized Document Embeddings Improve Topic Coherencecited by 121
  21. 2015Neural Variational Inference for Text Processingcited by 111
  22. 2017Discovering Discrete Latent Topics with Neural Variational Inferencecited by 79
  23. 2020Cross-lingual Contextualized Topic Models with Zero-shot Learningcited by 66
  24. 2022Making Science Simple: Corpora for the Lay Summarisation of Scientific Literaturecited by 54
  25. 2020Neural Topic Model via Optimal Transportcited by 50
  26. 2023Effective Neural Topic Modeling with Embedding Clustering Regularizationcited by 46
  27. 2019Topic-Guided Variational Autoencoders for Text Generationcited by 44
  28. 2022Mitigating Data Sparsity for Short Text Topic Modeling by Topic-Semantic Contrastive Learningcited by 38
  29. 2022Is Neural Topic Modelling Better than Clustering? An Empirical Study on Clustering with Contextual Embeddings for Topicscited by 35
  30. 2018Improving Topic Models with Latent Feature Word Representationscited by 28
  31. 2021Topic-Guided Abstractive Multi-Document Summarizationcited by 19
  32. 2023vONTSS: vMF based semi-supervised neural topic modeling with optimal transportcited by 16
  33. 2020Neural Topic Modeling by Incorporating Document Relationship Graphcited by 13
  34. 2020How Does a Neural Network's Architecture Impact Its Robustness to Noisy Labels?cited by 12
  35. 2019Topic Augmented Generator for Abstractive Summarizationcited by 6
  36. 2017Identifying Patterns of Associated-Conditions through Topic Models of Electronic Medical Recordscited by 5
  37. 2023Diversity-Aware Coherence Loss for Improving Neural Topic Modelscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 9; 2025: 6; 2026: 1.

16 papers.

Sort:

By area:

  1. 2024Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Surveycited by 157
  2. 2025Large Language Model (LLM) for Software Security: Code Analysis, Malware Analysis, Reverse Engineeringcited by 15
  3. 2024BERTrend: Neural Topic Modeling for Emerging Trends Detectioncited by 13
  4. 2024HR-MultiWOZ: A Task Oriented Dialogue (TOD) Dataset for HR LLM Agentcited by 12
  5. 2024Neural Topic Modeling with Large Language Models in the Loopcited by 10
  6. 2024Enhancing Short-Text Topic Modeling with LLM-Driven Context Expansion and Prefix-Tuned VAEscited by 5
  7. 2024Topic Classification of Case Law Using a Large Language Model and a New Taxonomy for UK Law: AI Insights into Summary Judgmentcited by 3
  8. 2025Quantifying Fairness in LLMs Beyond Tokens: A Semantic and Statistical Perspectivecited by 3
  9. 2025DeepSeek: Paradigm Shifts and Technical Evolution in Large AI Modelscited by 3
  10. 2024Topic Modeling with Fine-tuning LLMs and Bag of Sentencescited by 2
  11. 2024HR-Agent: A Task-Oriented Dialogue (TOD) LLM Agent Tailored for HR Applicationscited by 1
  12. 2025DiffETM: Diffusion Process Enhanced Embedded Topic Modelcited by 1
4 more
  1. 2025Evaluating Cognitive-Behavioral Fixation via Multimodal User Viewing Patterns on Social Mediacited by 1
  2. 2026LentEx: Generalizable Latent Entity Extraction via Synthetic Data and Instruction-Tuned LLMscited by 1
  3. 2024JADS: A Framework for Self-supervised Joint Aspect Discovery and Summarizationcited by 0
  4. 2025Understanding Cross-Domain Adaptation in Low-Resource Topic Modelingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Diffusion Models for Non-autoregressive Text Generation: A Survey
  2. 2025ClusterFusion: Hybrid Clustering with Embedding Guidance and LLM Adaptation
  3. 2022Latent Diffusion for Language Generation
  4. 2023A Survey of Diffusion Models in Natural Language Processing
  5. 2026How to Train Your Latent Diffusion Language Model Jointly With the Latent Space
  6. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Models
  7. 2025Diffusion vs. Autoregressive Language Models: A Text Embedding Perspective
  8. 2025Decoder-Only LLMs are Better Controllers for Diffusion Models