# Self-Supervised Transformer Architecture for Change Detection in Radio Access Networks

arXiv: https://arxiv.org/abs/2302.02025 · HTML: https://mlsanity.com/paper/2302.02025/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 38 references that are in the map.
- First posted on arXiv 3 Feb 2023.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [An Evaluation of Change Point Detection Algorithms](https://mlsanity.com/paper/2003.06222/) (2020)
- [Kernel Change-point Detection with Auxiliary Deep Generative Models](https://mlsanity.com/paper/1901.06077/) (2019)
- [A Nonparametric Approach for Multiple Change Point Analysis of Multivariate Data](https://mlsanity.com/paper/1306.4933/) (2013)
- [Change Point Detection in Time Series Data using Autoencoders with a Time-Invariant Representation](https://mlsanity.com/paper/2008.09524/) (2020)
- [Time Series Segmentation through Automatic Feature Learning](https://mlsanity.com/paper/1801.05394/) (2018)
- [Selective review of offline change point detection methods](https://mlsanity.com/paper/1801.00718/) (2018)
- [Deep RAN: A Scalable Data-driven platform to Detect Anomalies in Live Cellular Network Using Recurrent Convolutional Neural Network](https://mlsanity.com/paper/1911.04472/) (2019)

## Built on by (most cited first, 1 of 1)

- [Anomaly Detection for Scalable Task Grouping in Reinforcement Learning-based RAN Optimization](https://mlsanity.com/paper/2312.03277/) (2023)

## Abstract (arXiv)

Radio Access Networks (RANs) for telecommunications represent large agglomerations of interconnected hardware consisting of hundreds of thousands of transmitting devices (cells). Such networks undergo frequent and often heterogeneous changes caused by network operators, who are seeking to tune their system parameters for optimal performance. The effects of such changes are challenging to predict and will become even more so with the adoption of 5G/6G networks. Therefore, RAN monitoring is vital for network operators. We propose a self-supervised learning framework that leverages self-attention and self-distillation for this task. It works by detecting changes in Performance Measurement data, a collection of time-varying metrics which reflect a set of diverse measurements of the network performance at the cell level. Experimental results show that our approach outperforms the state of the art by 4% on a real-world based dataset consisting of about hundred thousands timeseries. It also has the merits of being scalable and generalizable. This allows it to provide deep insight into the specifics of mode of operation changes while relying minimally on expert knowledge.