# Narrative Shift Detection: A Hybrid Approach of Dynamic Topic Models and Large Language Models

arXiv: https://arxiv.org/abs/2506.20269 · HTML: https://mlsanity.com/paper/2506.20269/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 35 references that are in the map.
- First posted on arXiv 25 Jun 2025.

## Builds on (most cited first, 10 of 10)

- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks](https://mlsanity.com/paper/2303.15056/) (2023)
- [Can Large Language Models Transform Computational Social Science?](https://mlsanity.com/paper/2305.03514/) (2023)
- [Diachronic word embeddings and semantic shifts: a survey](https://mlsanity.com/paper/1806.03537/) (2018)
- [Can LLMs Learn Macroeconomic Narratives from Social Media?](https://mlsanity.com/paper/2406.12109/) (2024)
- [Zeitenwenden: Detecting changes in the German political discourse](https://mlsanity.com/paper/2410.17960/) (2024)
- [Predicting market inflation expectations with news topics and sentiment](https://mlsanity.com/paper/2107.07155/) (2021)

## Built on by (most cited first, 1 of 1)

- [Word-Centered Semantic Graphs for Interpretable Diachronic Sense Tracking](https://mlsanity.com/paper/2601.22410/) (2026)

## Abstract (arXiv)

With rapidly evolving media narratives, it has become increasingly critical to not just extract narratives from a given corpus but rather investigate, how they develop over time. While popular narrative extraction methods such as Large Language Models do well in capturing typical narrative elements or even the complex structure of a narrative, applying them to an entire corpus comes with obstacles, such as a high financial or computational cost. We propose a combination of the language understanding capabilities of Large Language Models with the large scale applicability of topic models to dynamically model narrative shifts across time using the Narrative Policy Framework. We apply a topic model and a corresponding change point detection method to find changes that concern a specific topic of interest. Using this model, we filter our corpus for documents that are particularly representative of that change and feed them into a Large Language Model that interprets the change that happened in an automated fashion and distinguishes between content and narrative shifts. We employ our pipeline on a corpus of The Wall Street Journal news paper articles from 2009 to 2023. Our findings indicate that a Large Language Model can efficiently extract a narrative shift if one exists at a given point in time, but does not perform as well when having to decide whether a shift in content or a narrative shift took place.