# Usage of specific attention improves change point detection

arXiv: https://arxiv.org/abs/2204.08175 · HTML: https://mlsanity.com/paper/2204.08175/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 17 references that are in the map.
- First posted on arXiv 18 Apr 2022.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [Efficient Transformers: A Survey](https://mlsanity.com/paper/2009.06732/) (2020)
- [Long Range Arena: A Benchmark for Efficient Transformers](https://mlsanity.com/paper/2011.04006/) (2020)
- [Real-world Anomaly Detection in Surveillance Videos](https://mlsanity.com/paper/1801.04264/) (2018)
- [An Evaluation of Change Point Detection Algorithms](https://mlsanity.com/paper/2003.06222/) (2020)
- [A Differentiable Language Model Adversarial Attack on Text Classifiers](https://mlsanity.com/paper/2107.11275/) (2021)
- [Online Neural Networks for Change-Point Detection](https://mlsanity.com/paper/2010.01388/) (2020)
- [Recurrent Convolutional Neural Networks help to predict location of Earthquakes](https://mlsanity.com/paper/2004.09140/) (2020)
- [Real-time data-driven detection of the rock type alteration during a directional drilling](https://mlsanity.com/paper/1903.11436/) (2019)
- [Detecting Performance Degradation of Software-Intensive Systems in the Presence of Trends and Long-Range Dependence](https://mlsanity.com/paper/1609.07662/) (2016)

## Abstract (arXiv)

The change point is a moment of an abrupt alteration in the data distribution. Current methods for change point detection are based on recurrent neural methods suitable for sequential data. However, recent works show that transformers based on attention mechanisms perform better than standard recurrent models for many tasks. The most benefit is noticeable in the case of longer sequences. In this paper, we investigate different attentions for the change point detection task and proposed specific form of attention related to the task at hand. We show that using a special form of attention outperforms state-of-the-art results.