# TLOB: A Novel Transformer Model with Dual Attention for Price Trend Prediction with Limit Order Book Data

arXiv: https://arxiv.org/abs/2502.15757 · HTML: https://mlsanity.com/paper/2502.15757/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 56 references that are in the map.
- First posted on arXiv 12 Feb 2025.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [Are Transformers Effective for Time Series Forecasting?](https://mlsanity.com/paper/2205.13504/) (2022)
- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second](https://mlsanity.com/paper/2207.01848/) (2022)
- [Transformers in Time Series: A Survey](https://mlsanity.com/paper/2202.07125/) (2022)
- [Symbolic Discovery of Optimization Algorithms](https://mlsanity.com/paper/2302.06675/) (2023)
- [Deep Reinforcement Learning for Trading](https://mlsanity.com/paper/1911.10107/) (2019)
- [Universal features of price formation in financial markets: perspectives from Deep Learning](https://mlsanity.com/paper/1803.06917/) (2018)
- [Benchmark Dataset for Mid-Price Forecasting of Limit Order Book Data with Machine Learning Methods](https://mlsanity.com/paper/1705.03233/) (2017)
- [Temporal Attention augmented Bilinear Network for Financial Time-Series Data Analysis](https://mlsanity.com/paper/1712.00975/) (2017)
- [Generating Realistic Stock Market Order Streams](https://mlsanity.com/paper/2006.04212/) (2020)
- [Towards Realistic Market Simulations: a Generative Adversarial Networks Approach](https://mlsanity.com/paper/2110.13287/) (2021)
- [Using Deep Learning for price prediction by exploiting stationary limit order book features](https://mlsanity.com/paper/1810.09965/) (2018)
- [Learning to simulate realistic limit order book markets from data as a World Agent](https://mlsanity.com/paper/2210.09897/) (2022)
- [Generative AI for End-to-End Limit Order Book Modelling: A Token-Level Autoregressive Generative Model of Message Flow Using a Deep State Space Network](https://mlsanity.com/paper/2309.00638/) (2023)
- [Data Normalization for Bilinear Structures in High-Frequency Financial Time-series](https://mlsanity.com/paper/2003.00598/) (2020)
- [Machine Learning for Forecasting Mid Price Movement using Limit Order Book Data](https://mlsanity.com/paper/1809.07861/) (2018)
- [Neural Stochastic Agent-Based Limit Order Book Simulation: A Hybrid Methodology](https://mlsanity.com/paper/2303.00080/) (2023)
- [Multi-Horizon Forecasting for Limit Order Books: Novel Deep Learning Approaches and Hardware Acceleration using Intelligent Processing Units](https://mlsanity.com/paper/2105.10430/) (2021)
- [LOB-Based Deep Learning Models for Stock Price Trend Prediction: A Benchmark Study](https://mlsanity.com/paper/2308.01915/) (2023)
- [TRADES: Generating Realistic Market Simulations with Diffusion Models](https://mlsanity.com/paper/2502.07071/) (2025)
- [Axial-LOB: High-Frequency Trading with Axial Attention](https://mlsanity.com/paper/2212.01807/) (2022)

## Built on by (most cited first, 4 of 4)

- [TradeFM: A Generative Foundation Model for Trade-flow and Market Microstructure](https://mlsanity.com/paper/2602.23784/) (2026)
- [LOBERT: Generative AI Foundation Model for Limit Order Book Messages](https://mlsanity.com/paper/2511.12563/) (2025)
- [Exploring Microstructural Dynamics in Cryptocurrency Limit Order Books: Better Inputs Matter More Than Stacking Another Hidden Layer](https://mlsanity.com/paper/2506.05764/) (2025)
- [Inference-Time Decision Calibration for Temporal Classification](https://mlsanity.com/paper/2606.16034/) (2026)

## Abstract (arXiv)

Price Trend Prediction (PTP) based on Limit Order Book (LOB) data is a fundamental challenge in financial markets. Despite advances in deep learning, existing models fail to generalize across different market conditions and assets. Surprisingly, by adapting a simple MLP-based architecture to LOB, we show that we surpass SoTA performance; thus, challenging the necessity of complex architectures. Unlike past work that shows robustness issues, we propose TLOB, a transformer-based model that uses a dual attention mechanism to capture spatial and temporal dependencies in LOB data. This allows it to adaptively focus on the market microstructure, making it particularly effective for longer-horizon predictions and volatile market conditions. We also introduce a new labeling method that improves on previous ones, removing the horizon bias. We evaluate TLOB's effectiveness across four horizons, using the established FI-2010 benchmark, a NASDAQ and a Bitcoin dataset. TLOB outperforms SoTA methods in every dataset and horizon. Additionally, we empirically show how stock price predictability has declined over time, -6.68 in F1-score, highlighting the growing market efficiency. Predictability must be considered in relation to transaction costs, so we experimented with defining trends using an average spread, reflecting the primary transaction cost. The resulting performance deterioration underscores the complexity of translating trend classification into profitable trading strategies. We argue that our work provides new insights into the evolving landscape of stock price trend prediction and sets a strong foundation for future advancements in financial AI. We release the code at https://github.com/LeonardoBerti00/TLOB.