Discovering Multi-Scale Semantic Structure in Text Corpora Using Density-Based Trees and LLM Embeddings
Not on the map. Why?
Abstract (arXiv)
Recent advances in large language models enable documents to be represented as dense semantic embeddings, supporting similarity-based operations over large text collections. However, many web-scale systems still rely on flat clustering or predefined taxonomies, limiting insight into hierarchical topic relationships. In this paper we operationalize hierarchical density modeling on large language model embeddings in a way not previously explored. Instead of enforcing a fixed taxonomy or single clustering resolution, the method progressively relaxes local density constraints, revealing how compact semantic groups merge into broader thematic regions. The resulting tree encodes multi-scale semantic organization directly from data, making structural relationships between topics explicit. We evaluate the hierarchies on standard text benchmarks, showing that semantic alignment peaks at intermediate density levels and that abrupt transitions correspond to meaningful changes in semantic resolution. Beyond benchmarks, the approach is applied to large institutional and scientific corpora, exposing dominant fields, cross-disciplinary proximities, and emerging thematic clusters. By framing hierarchical structure as an emergent property of density in embedding spaces, this method provides an interpretable, multi-scale representation of semantic structure suitable for large, evolving text collections.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 8 of its 34 references that are in the map. First posted on arXiv 29 Dec 2025.
Builds on
8 of 34 references.
References · 8
- 2017Attention Is All You Needcited by 60,361
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
- 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
- 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,386
- 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,261
- 2020Language-agnostic BERT Sentence Embeddingcited by 627
- 2025Evaluating NLP Embedding Models for Handling Science-Specific Symbolic Expressions in Student Textscited by 2
Built on by
Not cited yet.