# Advancing Text Classification with Large Language Models and Neural Attention Mechanisms

arXiv: https://arxiv.org/abs/2512.09444 · HTML: https://mlsanity.com/paper/2512.09444/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 22 references that are in the map.
- First posted on arXiv 10 Dec 2025.

## Builds on (most cited first, 10 of 10)

- [Text Classification via Large Language Models](https://mlsanity.com/paper/2305.08377/) (2023)
- [What Did I Do Wrong? Quantifying LLMs' Sensitivity and Consistency to Prompt Engineering](https://mlsanity.com/paper/2406.12334/) (2024)
- [Multi-Agent Reinforcement Learning for Adaptive Resource Orchestration in Cloud-Native Clusters](https://mlsanity.com/paper/2508.10253/) (2025)
- [Multi-class Text Classification using BERT-based Active Learning](https://mlsanity.com/paper/2104.14289/) (2021)
- [Evaluating Large Language Models for Health-Related Text Classification Tasks with Public Social Media Data](https://mlsanity.com/paper/2403.19031/) (2024)
- [Autonomous Resource Management in Microservice Systems via Reinforcement Learning](https://mlsanity.com/paper/2507.12879/) (2025)
- [Structural Priors and Modular Adapters in the Composable Fine-Tuning Algorithm of Large-Scale Models](https://mlsanity.com/paper/2511.03981/) (2025)
- [Dynamic Topic Evolution with Temporal Decay and Attention in Large Language Models](https://mlsanity.com/paper/2510.10613/) (2025)
- [LLM-Centric RAG with Multi-Granular Indexing and Confidence Constraints](https://mlsanity.com/paper/2510.27054/) (2025)
- [Contrastive Knowledge Transfer and Robust Optimization for Secure Alignment of Large Language Models](https://mlsanity.com/paper/2510.27077/) (2025)

## Built on by (most cited first, 1 of 1)

- [Coordinated Semantic Alignment and Evidence Constraints for Retrieval-Augmented Generation with Large Language Models](https://mlsanity.com/paper/2603.04647/) (2026)

## Abstract (arXiv)

This study proposes a text classification algorithm based on large language models, aiming to address the limitations of traditional methods in capturing long-range dependencies, understanding contextual semantics, and handling class imbalance. The framework includes text encoding, contextual representation modeling, attention-based enhancement, feature aggregation, and classification prediction. In the representation stage, deep semantic embeddings are obtained through large-scale pretrained language models, and attention mechanisms are applied to enhance the selective representation of key features. In the aggregation stage, global and weighted strategies are combined to generate robust text-level vectors. In the classification stage, a fully connected layer and Softmax output are used to predict class distributions, and cross-entropy loss is employed to optimize model parameters. Comparative experiments introduce multiple baseline models, including recurrent neural networks, graph neural networks, and Transformers, and evaluate them on Precision, Recall, F1-Score, and AUC. Results show that the proposed method outperforms existing models on all metrics, with especially strong improvements in Recall and AUC. In addition, sensitivity experiments are conducted on hyperparameters and data conditions, covering the impact of hidden dimensions on AUC and the impact of class imbalance ratios on Recall. The findings demonstrate that proper model configuration has a significant effect on performance and reveal the adaptability and stability of the model under different conditions. Overall, the proposed text classification method not only achieves effective performance improvement but also verifies its robustness and applicability in complex data environments through systematic analysis.