# Multi-head Temporal Attention-Augmented Bilinear Network for Financial time series prediction

arXiv: https://arxiv.org/abs/2201.05459 · HTML: https://mlsanity.com/paper/2201.05459/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 23 references that are in the map.
- First posted on arXiv 14 Jan 2022.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction](https://mlsanity.com/paper/1704.02971/) (2017)
- [Benchmark Dataset for Mid-Price Forecasting of Limit Order Book Data with Machine Learning Methods](https://mlsanity.com/paper/1705.03233/) (2017)
- [Temporal Attention augmented Bilinear Network for Financial Time-Series Data Analysis](https://mlsanity.com/paper/1712.00975/) (2017)
- [Data Normalization for Bilinear Structures in High-Frequency Financial Time-series](https://mlsanity.com/paper/2003.00598/) (2020)
- [Attention-based Neural Bag-of-Features Learning for Sequence Data](https://mlsanity.com/paper/2005.12250/) (2020)
- [Data-driven Neural Architecture Learning For Financial Time-series Forecasting](https://mlsanity.com/paper/1903.06751/) (2019)
- [Low-Rank Temporal Attention-Augmented Bilinear Network for financial time-series forecasting](https://mlsanity.com/paper/2107.06995/) (2021)

## Built on by (most cited first, 8 of 8)

- [HLOB -- Information Persistence and Structure in Limit Order Books](https://mlsanity.com/paper/2405.18938/) (2024)
- [Bayesian Bilinear Neural Network for Predicting the Mid-price Dynamics in Limit-Order Book Markets](https://mlsanity.com/paper/2203.03613/) (2022)
- [Exploiting Session Information in BERT-based Session-aware Sequential Recommendation](https://mlsanity.com/paper/2204.10851/) (2022)
- [Axial-LOB: High-Frequency Trading with Axial Attention](https://mlsanity.com/paper/2212.01807/) (2022)
- [Predicting the State of Synchronization of Financial Time Series using Cross Recurrence Plots](https://mlsanity.com/paper/2210.14605/) (2022)
- [Quantile Online Learning for Semiconductor Failure Analysis](https://mlsanity.com/paper/2303.07062/) (2023)
- [Market Making with Deep Reinforcement Learning from Limit Order Books](https://mlsanity.com/paper/2305.15821/) (2023)
- [Leveraging Deep Learning and Online Source Sentiment for Financial Portfolio Management](https://mlsanity.com/paper/2309.16679/) (2023)

## Abstract (arXiv)

Financial time-series forecasting is one of the most challenging domains in the field of time-series analysis. This is mostly due to the highly non-stationary and noisy nature of financial time-series data. With progressive efforts of the community to design specialized neural networks incorporating prior domain knowledge, many financial analysis and forecasting problems have been successfully tackled. The temporal attention mechanism is a neural layer design that recently gained popularity due to its ability to focus on important temporal events. In this paper, we propose a neural layer based on the ideas of temporal attention and multi-head attention to extend the capability of the underlying neural network in focusing simultaneously on multiple temporal instances. The effectiveness of our approach is validated using large-scale limit-order book market data to forecast the direction of mid-price movements. Our experiments show that the use of multi-head temporal attention modules leads to enhanced prediction performances compared to baseline models.