# Financial Time Series Forecasting using CNN and Transformer

arXiv: https://arxiv.org/abs/2304.04912 · HTML: https://mlsanity.com/paper/2304.04912/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 24 references that are in the map.
- First posted on arXiv 11 Apr 2023.

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model](https://mlsanity.com/paper/2201.11990/) (2022)

## Built on by (most cited first, 15 of 15)

- [Population Aware Diffusion for Time Series Generation](https://mlsanity.com/paper/2501.00910/) (2025)
- [TradExpert: Revolutionizing Trading with Mixture of Expert LLMs](https://mlsanity.com/paper/2411.00782/) (2024)
- [LSR-IGRU: Stock Trend Prediction Based on Long Short-Term Relationships and Improved GRU](https://mlsanity.com/paper/2409.08282/) (2024)
- [From Pixels to Predictions: Spectrogram and Vision Transformer for Better Time Series Forecasting](https://mlsanity.com/paper/2403.11047/) (2024)
- [Quantformer: from attention to profit with a quantitative transformer trading strategy](https://mlsanity.com/paper/2404.00424/) (2024)
- [MCI-GRU: Stock Prediction Model Based on Multi-Head Cross-Attention and Improved GRU](https://mlsanity.com/paper/2410.20679/) (2024)
- [A quantitative fusion strategy of stock picking and timing based on Particle Swarm Optimized-Back Propagation Neural Network and Multivariate Gaussian-Hidden Markov Model](https://mlsanity.com/paper/2312.05756/) (2023)
- [SwiftQueue: Optimizing Low-Latency Applications with Swift Packet Queuing](https://mlsanity.com/paper/2410.06112/) (2024)
- [A Novel Loss Function for Deep Learning Based Daily Stock Trading System](https://mlsanity.com/paper/2502.17493/) (2025)
- [Temporal Window Smoothing of Exogenous Variables for Improved Time Series Prediction](https://mlsanity.com/paper/2507.05284/) (2025)
- [TABL-ABM: A Hybrid Framework for Synthetic LOB Generation](https://mlsanity.com/paper/2510.22685/) (2025)
- [Towards Explainable and Reliable AI in Finance](https://mlsanity.com/paper/2510.26353/) (2025)
- [An Interpretable Generative Framework for Anomaly Detection in High-Dimensional Financial Time Series](https://mlsanity.com/paper/2603.07864/) (2026)
- [ARTEMIS: A Neuro Symbolic Framework for Economically Constrained Market Dynamics](https://mlsanity.com/paper/2603.18107/) (2026)
- [Emergent Latent-State Computation under Stochastic Volatility](https://mlsanity.com/paper/2607.25459/) (2026)

## Abstract (arXiv)

Time series forecasting is important across various domains for decision-making. In particular, financial time series such as stock prices can be hard to predict as it is difficult to model short-term and long-term temporal dependencies between data points. Convolutional Neural Networks (CNN) are good at capturing local patterns for modeling short-term dependencies. However, CNNs cannot learn long-term dependencies due to the limited receptive field. Transformers on the other hand are capable of learning global context and long-term dependencies. In this paper, we propose to harness the power of CNNs and Transformers to model both short-term and long-term dependencies within a time series, and forecast if the price would go up, down or remain the same (flat) in the future. In our experiments, we demonstrated the success of the proposed method in comparison to commonly adopted statistical and deep learning methods on forecasting intraday stock price change of S&P 500 constituents.