# Multi-Transformer: A New Neural Network-Based Architecture for Forecasting S&P Volatility

arXiv: https://arxiv.org/abs/2109.12621 · HTML: https://mlsanity.com/paper/2109.12621/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 76 references that are in the map.
- First posted on arXiv 26 Sep 2021.

## Builds on (most cited first, 5 of 5)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)

## Built on by (most cited first, 10 of 10)

- [Transformer-Based Deep Learning Model for Stock Price Prediction: A Case Study on Bangladesh Stock Market](https://mlsanity.com/paper/2208.08300/) (2022)
- [DeepVol: Volatility Forecasting from High-Frequency Data with Dilated Causal Convolutions](https://mlsanity.com/paper/2210.04797/) (2022)
- [Quantformer: from attention to profit with a quantitative transformer trading strategy](https://mlsanity.com/paper/2404.00424/) (2024)
- [Forecasting Bitcoin volatility spikes from whale transactions and CryptoQuant data using Synthesizer Transformer models](https://mlsanity.com/paper/2211.08281/) (2022)
- [From GARCH to Neural Network for Volatility Forecast](https://mlsanity.com/paper/2402.06642/) (2024)
- [Trading through Earnings Seasons using Self-Supervised Contrastive Representation Learning](https://mlsanity.com/paper/2409.17392/) (2024)
- [Adaptive Temporal Fusion Transformers for Cryptocurrency Price Prediction](https://mlsanity.com/paper/2509.10542/) (2025)
- [Multivariate Realized Volatility Forecasting with Graph Neural Network](https://mlsanity.com/paper/2112.09015/) (2021)
- [Uniform Ck Approximation of G-Invariant and Antisymmetric Functions, Embedding Dimensions, and Polynomial Representations](https://mlsanity.com/paper/2403.01339/) (2024)
- [A Hybrid Quantum Circuit Born Machine Framework for Financial Volatility Forecasting: Quantum-Assisted Training and Classical Inference](https://mlsanity.com/paper/2603.09789/) (2026)

## Abstract (arXiv)

Events such as the Financial Crisis of 2007-2008 or the COVID-19 pandemic caused significant losses to banks and insurance entities. They also demonstrated the importance of using accurate equity risk models and having a risk management function able to implement effective hedging strategies. Stock volatility forecasts play a key role in the estimation of equity risk and, thus, in the management actions carried out by financial institutions. Therefore, this paper has the aim of proposing more accurate stock volatility models based on novel machine and deep learning techniques. This paper introduces a neural network-based architecture, called Multi-Transformer. Multi-Transformer is a variant of Transformer models, which have already been successfully applied in the field of natural language processing. Indeed, this paper also adapts traditional Transformer layers in order to be used in volatility forecasting models. The empirical results obtained in this paper suggest that the hybrid models based on Multi-Transformer and Transformer layers are more accurate and, hence, they lead to more appropriate risk measures than other autoregressive algorithms or hybrid models based on feed forward layers or long short term memory cells.