# Dynamic Topic Evolution with Temporal Decay and Attention in Large Language Models

arXiv: https://arxiv.org/abs/2510.10613 · HTML: https://mlsanity.com/paper/2510.10613/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 26 references that are in the map.
- First posted on arXiv 12 Oct 2025.

## Builds on (most cited first, 12 of 12)

- [DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing](https://mlsanity.com/paper/2111.09543/) (2021)
- [Modeling Dynamic Topics in Chain-Free Fashion by Evolution-Tracking Contrastive Learning and Unassociated Word Exclusion](https://mlsanity.com/paper/2405.17957/) (2024)
- [Structure-Learnable Adapter Fine-Tuning for Parameter-Efficient Large Language Models](https://mlsanity.com/paper/2509.03057/) (2025)
- [Neural Dynamic Focused Topic Model](https://mlsanity.com/paper/2301.10988/) (2023)
- [High Fidelity Text to Image Generation with Contrastive Alignment and Structural Guidance](https://mlsanity.com/paper/2508.10280/) (2025)
- [TempoFormer: A Transformer for Temporally-aware Representations in Change Detection](https://mlsanity.com/paper/2408.15689/) (2024)
- [Semantic and Structural Analysis of Implicit Biases in Large Language Models: An Interpretable Approach](https://mlsanity.com/paper/2508.06155/) (2025)
- [Knowledge Graph-Infused Fine-Tuning for Structured Reasoning in Large Language Models](https://mlsanity.com/paper/2508.14427/) (2025)
- [Graph Neural Network and Transformer Integration for Unsupervised System Anomaly Discovery](https://mlsanity.com/paper/2508.09401/) (2025)
- [Integrating Feature Attention and Temporal Modeling for Collaborative Financial Risk Assessment](https://mlsanity.com/paper/2508.09399/) (2025)
- [Time-aware topic identification in social media with pre-trained language models: A case study of electric vehicles](https://mlsanity.com/paper/2210.05143/) (2022)
- [Topology-Aware Graph Reinforcement Learning for Dynamic Routing in Cloud Networks](https://mlsanity.com/paper/2509.04973/) (2025)

## Built on by (most cited first, 3 of 3)

- [Knowledge-Augmented Large Language Model Agents for Explainable Financial Decision-Making](https://mlsanity.com/paper/2512.09440/) (2025)
- [Structure-Aware Decoding Mechanisms for Complex Entity Extraction with Large-Scale Language Models](https://mlsanity.com/paper/2512.13980/) (2025)
- [Advancing Text Classification with Large Language Models and Neural Attention Mechanisms](https://mlsanity.com/paper/2512.09444/) (2025)

## Abstract (arXiv)

This paper proposes a modeling framework for dynamic topic evolution based on temporal large language models. The method first uses a large language model to obtain contextual embeddings of text and then introduces a temporal decay function and an attention mechanism. These components allow the model to adjust the importance of semantic units according to time intervals and capture topic variations across different periods. The temporal representations are then mapped into a latent topic space, where a state transition matrix is applied to describe the dynamic evolution of topics. A joint optimization objective constrains both semantic modeling and temporal consistency, ensuring diversity and smoothness in topic generation. The design emphasizes the unified modeling of semantic representation and temporal evolution, which improves topic coherence and diversity while enhancing stability and interpretability over time. Experiments on real-world corpora show that the framework effectively captures the generation, expansion, and decline of topics and outperforms existing models across multiple metrics. Overall, the proposed method provides a systematic solution for understanding dynamic semantic patterns in large-scale text, enriches the research paradigm of topic modeling, and supports complex text analysis tasks in multiple domains.