# Multi-Horizon Forecasting for Limit Order Books: Novel Deep Learning Approaches and Hardware Acceleration using Intelligent Processing Units

arXiv: https://arxiv.org/abs/2105.10430 · HTML: https://mlsanity.com/paper/2105.10430/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 39 references that are in the map.
- First posted on arXiv 21 May 2021.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Effective Approaches to Attention-based Neural Machine Translation](https://mlsanity.com/paper/1508.04025/) (2015)
- [TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems](https://mlsanity.com/paper/1603.04467/) (2016)
- [Deep Reinforcement Learning for Trading](https://mlsanity.com/paper/1911.10107/) (2019)
- [Universal features of price formation in financial markets: perspectives from Deep Learning](https://mlsanity.com/paper/1803.06917/) (2018)
- [Benchmark Dataset for Mid-Price Forecasting of Limit Order Book Data with Machine Learning Methods](https://mlsanity.com/paper/1705.03233/) (2017)
- [Dissecting the Graphcore IPU Architecture via Microbenchmarking](https://mlsanity.com/paper/1912.03413/) (2019)
- [Temporal Attention augmented Bilinear Network for Financial Time-Series Data Analysis](https://mlsanity.com/paper/1712.00975/) (2017)
- [Using Deep Learning for price prediction by exploiting stationary limit order book features](https://mlsanity.com/paper/1810.09965/) (2018)
- [Deep Reinforcement Learning for Active High Frequency Trading](https://mlsanity.com/paper/2101.07107/) (2021)
- [Model-based Reinforcement Learning for Predictions and Control for Limit Order Books](https://mlsanity.com/paper/1910.03743/) (2019)
- [Deep Learning modeling of Limit Order Book: a comparative perspective](https://mlsanity.com/paper/2007.07319/) (2020)
- [Deep Learning for Market by Order Data](https://mlsanity.com/paper/2102.08811/) (2021)
- [DeepFolio: Convolutional Neural Networks for Portfolios with Limit Order Book Data](https://mlsanity.com/paper/2008.12152/) (2020)

## Built on by (most cited first, 10 of 10)

- [LOB-Based Deep Learning Models for Stock Price Trend Prediction: A Benchmark Study](https://mlsanity.com/paper/2308.01915/) (2023)
- [Realised Volatility Forecasting: Machine Learning via Financial Word Embedding](https://mlsanity.com/paper/2108.00480/) (2021)
- [Deep Limit Order Book Forecasting](https://mlsanity.com/paper/2403.09267/) (2024)
- [Axial-LOB: High-Frequency Trading with Axial Attention](https://mlsanity.com/paper/2212.01807/) (2022)
- [Transformers versus LSTMs for electronic trading](https://mlsanity.com/paper/2309.11400/) (2023)
- [TLOB: A Novel Transformer Model with Dual Attention for Price Trend Prediction with Limit Order Book Data](https://mlsanity.com/paper/2502.15757/) (2025)
- [Asynchronous Deep Double Duelling Q-Learning for Trading-Signal Execution in Limit Order Book Markets](https://mlsanity.com/paper/2301.08688/) (2023)
- [Market Making with Deep Reinforcement Learning from Limit Order Books](https://mlsanity.com/paper/2305.15821/) (2023)
- [NanoBatch Privacy: Enabling fast Differentially Private learning on the IPU](https://mlsanity.com/paper/2109.12191/) (2021)
- [Accelerating Machine Learning Training Time for Limit Order Book Prediction](https://mlsanity.com/paper/2206.09041/) (2022)

## Abstract (arXiv)

We design multi-horizon forecasting models for limit order book (LOB) data by using deep learning techniques. Unlike standard structures where a single prediction is made, we adopt encoder-decoder models with sequence-to-sequence and Attention mechanisms to generate a forecasting path. Our methods achieve comparable performance to state-of-art algorithms at short prediction horizons. Importantly, they outperform when generating predictions over long horizons by leveraging the multi-horizon setup. Given that encoder-decoder models rely on recurrent neural layers, they generally suffer from slow training processes. To remedy this, we experiment with utilising novel hardware, so-called Intelligent Processing Units (IPUs) produced by Graphcore. IPUs are specifically designed for machine intelligence workload with the aim to speed up the computation process. We show that in our setup this leads to significantly faster training times when compared to training models with GPUs.