# Predicting Stock Movement with BERTweet and Transformers

arXiv: https://arxiv.org/abs/2503.10957 · HTML: https://mlsanity.com/paper/2503.10957/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 13 references that are in the map.
- First posted on arXiv 13 Mar 2025.

## Builds on (most cited first, 6 of 6)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BERTweet: A pre-trained language model for English Tweets](https://mlsanity.com/paper/2005.10200/) (2020)
- [Listening to Chaotic Whispers: A Deep Learning Framework for News-oriented Stock Trend Prediction](https://mlsanity.com/paper/1712.02136/) (2017)
- [HATS: A Hierarchical Graph Attention Network for Stock Movement Prediction](https://mlsanity.com/paper/1908.07999/) (2019)
- [Stock Movement Prediction with Multimodal Stable Fusion via Gated Cross-Attention Mechanism](https://mlsanity.com/paper/2406.06594/) (2024)

## Abstract (arXiv)

Applying deep learning and computational intelligence to finance has been a popular area of applied research, both within academia and industry, and continues to attract active attention. The inherently high volatility and non-stationary of the data pose substantial challenges to machine learning models, especially so for today's expressive and highly-parameterized deep learning models. Recent work has combined natural language processing on data from social media to augment models based purely on historic price data to improve performance has received particular attention. Previous work has achieved state-of-the-art performance on this task by combining techniques such as bidirectional GRUs, variational autoencoders, word and document embeddings, self-attention, graph attention, and adversarial training. In this paper, we demonstrated the efficacy of BERTweet, a variant of BERT pre-trained specifically on a Twitter corpus, and the transformer architecture by achieving competitive performance with the existing literature and setting a new baseline for Matthews Correlation Coefficient on the Stocknet dataset without auxiliary data sources.