mlsanityData to 14 Oct 2026

Beyond Top Words: MonoTM for Topic Modeling with Interpretable Monosemantic Features

arXiv 2609.09575 ↗First posted 9 Sep 2026cs.CL cs.HCcited by 0

Not on the map. Why?

Abstract (arXiv)

Topic models summarize large text corpora, but top-ranked words often provide only a limited representation of topic semantics. Sparse autoencoders (SAEs) offer a way to move beyond word-level descriptors by extracting interpretable features from dense representations, yet how feature interpretability relates to topic-inference quality remains unclear. We introduce MonoTM, an interpretable topic modeling framework that decouples these roles. Across three benchmark corpora, we show that document--topic mixture estimation and semantic interpretation favor different SAE configurations and feature subsets. MonoTM estimates mixtures from the full SAE bag-of-features representation and, with them fixed, learns topic descriptors over a separate vocabulary of corpus-grounded semantic features. This design preserves global topic structure while representing topics with semantic units more meaningful than individual words, making them more useful for downstream corpus analysis.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 14 of its 36 references that are in the map. First posted on arXiv 9 Sep 2026.

Builds on

14 of 36 references.

References · 14

  1. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  2. 2022BERTopic: Neural topic modeling with a class-based TF-IDF procedurecited by 801
  3. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 394
  4. 2019Topic Modeling in Embedding Spacescited by 224
  5. 2017Autoencoding Variational Inference For Topic Modelscited by 169
  6. 2020Pre-training is a Hot Topic: Contextualized Document Embeddings Improve Topic Coherencecited by 121
  7. 2017HDLTex: Hierarchical Deep Learning for Text Classificationcited by 106
  8. 2017Discovering Discrete Latent Topics with Neural Variational Inferencecited by 79
  9. 2024Concept Induction: Analyzing Unstructured Text with High-Level Concepts Using LLooMcited by 78
  10. 2020Cross-lingual Contextualized Topic Models with Zero-shot Learningcited by 67
  11. 2025Llama-Embed-Nemotron-8B: A Universal Text Embedding Model for Multilingual and Cross-Lingual Taskscited by 51
  12. 2023Effective Neural Topic Modeling with Embedding Clustering Regularizationcited by 46
2 more
  1. 2024FASTopic: Pretrained Transformer is a Fast, Adaptive, Stable, and Transferable Topic Modelcited by 35
  2. 2025Sparse Autoencoders are Topic Modelscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Model Directions, Not Words: Mechanistic Topic Models Using Sparse Autoencoders
  2. 2024Disentangling Dense Embeddings with Sparse Autoencoders
  3. 2025Interpretable Embeddings with Sparse Autoencoders: A Data Analysis Toolkit
  4. 2023Deep de Finetti: Recovering Topic Distributions from Large Language Models
  5. 2025Decoding Dense Embeddings: Sparse Autoencoders for Interpreting and Discretizing Dense Retrieval
  6. 2024Addressing Topic Granularity and Hallucination in Large Language Models for Topic Modelling
  7. 2025Unveiling Decision-Making in LLMs for Text Classification : Extraction of influential and interpretable concepts with Sparse Autoencoders
  8. 2026LLM-XTM: Enhancing Cross-Lingual Topic Models with Large Language Models