# Contextual Categorization Enhancement through LLMs Latent-Space

arXiv: https://arxiv.org/abs/2404.16442 · HTML: https://mlsanity.com/paper/2404.16442/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 17 references that are in the map.
- First posted on arXiv 25 Apr 2024.

## Builds on (most cited first, 6 of 6)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs](https://mlsanity.com/paper/1603.09320/) (2016)
- [Pairwise Multi-Class Document Classification for Semantic Relations between Wikipedia Articles](https://mlsanity.com/paper/2003.09881/) (2020)
- [Language-agnostic Topic Classification for Wikipedia](https://mlsanity.com/paper/2103.00068/) (2021)
- [The Use of NLP-Based Text Representation Techniques to Support Requirement Engineering Tasks: A Systematic Mapping Review](https://mlsanity.com/paper/2206.00421/) (2022)
- [Mapping Researcher Activity based on Publication Data by means of Transformers](https://mlsanity.com/paper/2306.09049/) (2023)

## Built on by (most cited first, 2 of 2)

- [Leveraging Taxonomy and LLMs for Improved Multimodal Hierarchical Classification](https://mlsanity.com/paper/2501.06827/) (2025)
- [Enforcing Consistency and Fairness in Multi-level Hierarchical Classification with a Mask-based Output Layer](https://mlsanity.com/paper/2503.15566/) (2025)

## Abstract (arXiv)

Managing the semantic quality of the categorization in large textual datasets, such as Wikipedia, presents significant challenges in terms of complexity and cost. In this paper, we propose leveraging transformer models to distill semantic information from texts in the Wikipedia dataset and its associated categories into a latent space. We then explore different approaches based on these encodings to assess and enhance the semantic identity of the categories. Our graphical approach is powered by Convex Hull, while we utilize Hierarchical Navigable Small Worlds (HNSWs) for the hierarchical approach. As a solution to the information loss caused by the dimensionality reduction, we modulate the following mathematical solution: an exponential decay function driven by the Euclidean distances between the high-dimensional encodings of the textual categories. This function represents a filter built around a contextual category and retrieves items with a certain Reconsideration Probability (RP). Retrieving high-RP items serves as a tool for database administrators to improve data groupings by providing recommendations and identifying outliers within a contextual framework.