# ViT-CAT: Parallel Vision Transformers with Cross Attention Fusion for Popularity Prediction in MEC Networks

arXiv: https://arxiv.org/abs/2210.15125 · HTML: https://mlsanity.com/paper/2210.15125/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 38 references that are in the map.
- First posted on arXiv 27 Oct 2022.

## Builds on (most cited first, 8 of 8)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Edge-Computing-Enabled Smart Cities: A Comprehensive Survey](https://mlsanity.com/paper/1909.08747/) (2019)
- [Deep Learning Based Caching for Self-Driving Car in Multi-access Edge Computing](https://mlsanity.com/paper/1810.01548/) (2018)
- [TEDGE-Caching: Transformer-based Edge Caching Towards 6G Networks](https://mlsanity.com/paper/2112.00633/) (2021)
- [Deep Learning for Wireless Coded Caching with Unknown and Time-Variant Content Popularity](https://mlsanity.com/paper/2008.09422/) (2020)
- [Joint Transmission Scheme and Coded Content Placement in Cluster-centric UAV-aided Cellular Networks](https://mlsanity.com/paper/2101.11787/) (2021)
- [Multi-Content Time-Series Popularity Prediction with Multiple-Model Transformers in MEC Networks](https://mlsanity.com/paper/2210.05874/) (2022)

## Built on by (most cited first, 1 of 1)

- [CLSA: Contrastive Learning-based Survival Analysis for Popularity Prediction in MEC Networks](https://mlsanity.com/paper/2303.12097/) (2023)

## Abstract (arXiv)

Mobile Edge Caching (MEC) is a revolutionary technology for the Sixth Generation (6G) of wireless networks with the promise to significantly reduce users' latency via offering storage capacities at the edge of the network. The efficiency of the MEC network, however, critically depends on its ability to dynamically predict/update the storage of caching nodes with the top-K popular contents. Conventional statistical caching schemes are not robust to the time-variant nature of the underlying pattern of content requests, resulting in a surge of interest in using Deep Neural Networks (DNNs) for time-series popularity prediction in MEC networks. However, existing DNN models within the context of MEC fail to simultaneously capture both temporal correlations of historical request patterns and the dependencies between multiple contents. This necessitates an urgent quest to develop and design a new and innovative popularity prediction architecture to tackle this critical challenge. The paper addresses this gap by proposing a novel hybrid caching framework based on the attention mechanism. Referred to as the parallel Vision Transformers with Cross Attention (ViT-CAT) Fusion, the proposed architecture consists of two parallel ViT networks, one for collecting temporal correlation, and the other for capturing dependencies between different contents. Followed by a Cross Attention (CA) module as the Fusion Center (FC), the proposed ViT-CAT is capable of learning the mutual information between temporal and spatial correlations, as well, resulting in improving the classification accuracy, and decreasing the model's complexity about 8 times. Based on the simulation results, the proposed ViT-CAT architecture outperforms its counterparts across the classification accuracy, complexity, and cache-hit ratio.