# StonkBERT: Can Language Models Predict Medium-Run Stock Price Movements?

arXiv: https://arxiv.org/abs/2202.02268 · HTML: https://mlsanity.com/paper/2202.02268/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 29 references that are in the map.
- First posted on arXiv 4 Feb 2022.

## Builds on (most cited first, 9 of 9)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [FinBERT: Financial Sentiment Analysis with Pre-trained Language Models](https://mlsanity.com/paper/1908.10063/) (2019)
- [German's Next Language Model](https://mlsanity.com/paper/2010.10906/) (2020)
- [Comparing BERT against traditional machine learning text classification](https://mlsanity.com/paper/2005.13012/) (2020)
- [Stock Movement Prediction with Financial News using Contextualized Embedding from BERT](https://mlsanity.com/paper/2107.08721/) (2021)
- [Stock price prediction using BERT and GAN](https://mlsanity.com/paper/2107.09055/) (2021)
- [Multi-Domain Transformer-Based Counterfactual Augmentation for Earnings Call Analysis](https://mlsanity.com/paper/2112.00963/) (2021)

## Built on by (most cited first, 1 of 1)

- [Multilingual De-Duplication Strategies: Applying scalable similarity search with monolingual & multilingual embedding models](https://mlsanity.com/paper/2406.13695/) (2024)

## Abstract (arXiv)

To answer this question, we fine-tune transformer-based language models, including BERT, on different sources of company-related text data for a classification task to predict the one-year stock price performance. We use three different types of text data: News articles, blogs, and annual reports. This allows us to analyze to what extent the performance of language models is dependent on the type of the underlying document. StonkBERT, our transformer-based stock performance classifier, shows substantial improvement in predictive accuracy compared to traditional language models. The highest performance was achieved with news articles as text source. Performance simulations indicate that these improvements in classification accuracy also translate into above-average stock market returns.