# Neural Total Variation Distance Estimators for Changepoint Detection in News Data

arXiv: https://arxiv.org/abs/2506.18764 · HTML: https://mlsanity.com/paper/2506.18764/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 56 references that are in the map.
- First posted on arXiv 23 Jun 2025.

## Builds on (most cited first, 23 of 23)

- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization](https://mlsanity.com/paper/1606.00709/) (2016)
- [Breaking the Curse of Horizon: Infinite-Horizon Off-Policy Estimation](https://mlsanity.com/paper/1810.12429/) (2018)
- [Telescoping Density-Ratio Estimation](https://mlsanity.com/paper/2006.12204/) (2020)
- [Statistically Significant Detection of Linguistic Change](https://mlsanity.com/paper/1411.3315/) (2014)
- [An Evaluation of Change Point Detection Algorithms](https://mlsanity.com/paper/2003.06222/) (2020)
- [Time Series Change Point Detection with Self-Supervised Contrastive Predictive Coding](https://mlsanity.com/paper/2011.14097/) (2020)
- [Featurized Density Ratio Estimation](https://mlsanity.com/paper/2107.02212/) (2021)
- [Change Point Detection in Time Series Data using Autoencoders with a Time-Invariant Representation](https://mlsanity.com/paper/2008.09524/) (2020)
- [Training Neural Networks for Likelihood/Density Ratio Estimation](https://mlsanity.com/paper/1911.00405/) (2019)
- [Quantum chi-squared tomography and mutual information testing](https://mlsanity.com/paper/2305.18519/) (2023)
- [Machine learning phase transitions: Connections to the Fisher information](https://mlsanity.com/paper/2311.10710/) (2023)
- [Tracking changes using Kullback-Leibler divergence for the continual learning](https://mlsanity.com/paper/2210.04865/) (2022)
- [Real-time financial surveillance via quickest change-point detection methods](https://mlsanity.com/paper/1509.01570/) (2015)
- [Deep Learning for Multi-Scale Changepoint Detection in Multivariate Time Series](https://mlsanity.com/paper/1905.06913/) (2019)
- [Deep density ratio estimation for change point detection](https://mlsanity.com/paper/1905.09876/) (2019)
- [High-Dimensional Changepoint Detection via a Geometrically Inspired Mapping](https://mlsanity.com/paper/2001.05241/) (2020)
- [High-dimensional Change-point Detection Using Generalized Homogeneity Metrics](https://mlsanity.com/paper/2105.08976/) (2021)
- [Discovering collective narratives shifts in online discussions](https://mlsanity.com/paper/2307.08541/) (2023)
- [Fast Detection of Phase Transitions with Multi-Task Learning-by-Confusion](https://mlsanity.com/paper/2311.09128/) (2023)
- [Continual Density Ratio Estimation in an Online Setting](https://mlsanity.com/paper/2103.05276/) (2021)
- [A changepoint approach to modelling non-stationary soil moisture dynamics](https://mlsanity.com/paper/2310.17546/) (2023)
- [A Comparison of Single and Multiple Changepoint Techniques for Time Series Data](https://mlsanity.com/paper/2101.01960/) (2021)

## Abstract (arXiv)

Detecting when public discourse shifts in response to major events is crucial for understanding societal dynamics. Real-world data is high-dimensional, sparse, and noisy, making changepoint detection in this domain a challenging endeavor. In this paper, we leverage neural networks for changepoint detection in news data, introducing a method based on the so-called learning-by-confusion scheme, which was originally developed for detecting phase transitions in physical systems. We train classifiers to distinguish between articles from different time periods. The resulting classification accuracy is used to estimate the total variation distance between underlying content distributions, where significant distances highlight changepoints. We demonstrate the effectiveness of this method on both synthetic datasets and real-world data from The Guardian newspaper, successfully identifying major historical events including 9/11, the COVID-19 pandemic, and presidential elections. Our approach requires minimal domain knowledge, can autonomously discover significant shifts in public discourse, and yields a quantitative measure of change in content, making it valuable for journalism, policy analysis, and crisis monitoring.