# PreBit -- A multimodal model with Twitter FinBERT embeddings for extreme price movement prediction of Bitcoin

arXiv: https://arxiv.org/abs/2206.00648 · HTML: https://mlsanity.com/paper/2206.00648/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 94 references that are in the map.
- First posted on arXiv 30 May 2022.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [FinBERT: Financial Sentiment Analysis with Pre-trained Language Models](https://mlsanity.com/paper/1908.10063/) (2019)
- [Utilizing BERT for Aspect-Based Sentiment Analysis via Constructing Auxiliary Sentence](https://mlsanity.com/paper/1903.09588/) (2019)
- [Sentiment Analysis of Twitter Data for Predicting Stock Market Movements](https://mlsanity.com/paper/1610.09225/) (2016)
- [Stock Movement Prediction with Financial News using Contextualized Embedding from BERT](https://mlsanity.com/paper/2107.08721/) (2021)
- [Stock price prediction using BERT and GAN](https://mlsanity.com/paper/2107.09055/) (2021)
- [Text Mining of Stocktwits Data for Predicting Stock Prices](https://mlsanity.com/paper/2103.16388/) (2021)
- [Real-Time Prediction of BITCOIN Price using Machine Learning Techniques and Public Sentiment Analysis](https://mlsanity.com/paper/2006.14473/) (2020)
- [KryptoOracle: A Real-Time Cryptocurrency Price Prediction Platform Using Twitter Sentiments](https://mlsanity.com/paper/2003.04967/) (2020)
- [MGNC-CNN: A Simple Approach to Exploiting Multiple Word Embeddings for Sentence Classification](https://mlsanity.com/paper/1603.00968/) (2016)
- [Bitcoin Transaction Strategy Construction Based on Deep Reinforcement Learning](https://mlsanity.com/paper/2109.14789/) (2021)
- [Ask "Who", Not "What": Bitcoin Volatility Forecasting with Twitter Data](https://mlsanity.com/paper/2110.14317/) (2021)

## Built on by (most cited first, 7 of 7)

- [Constructing Time-Series Momentum Portfolios with Deep Multi-Task Learning](https://mlsanity.com/paper/2306.13661/) (2023)
- [Forecasting Bitcoin volatility spikes from whale transactions and CryptoQuant data using Synthesizer Transformer models](https://mlsanity.com/paper/2211.08281/) (2022)
- [Dynamic Bayesian Networks for Predicting Cryptocurrency Price Directions: Uncovering Causal Relationships](https://mlsanity.com/paper/2306.08157/) (2023)
- [Incorporating Pre-trained Model Prompting in Multimodal Stock Volume Movement Prediction](https://mlsanity.com/paper/2309.05608/) (2023)
- [Market-Derived Financial Sentiment Analysis: Context-Aware Language Models for Crypto Forecasting](https://mlsanity.com/paper/2502.14897/) (2025)
- [Smart Timing for Mining: A Deep Learning Framework for Bitcoin Hardware ROI Prediction](https://mlsanity.com/paper/2512.05402/) (2025)
- [CGCMA: Conditionally-Gated Cross-Modal Attention for Event-Conditioned Asynchronous Fusion](https://mlsanity.com/paper/2604.16411/) (2026)

## Abstract (arXiv)

Bitcoin, with its ever-growing popularity, has demonstrated extreme price volatility since its origin. This volatility, together with its decentralised nature, make Bitcoin highly subjective to speculative trading as compared to more traditional assets. In this paper, we propose a multimodal model for predicting extreme price fluctuations. This model takes as input a variety of correlated assets, technical indicators, as well as Twitter content. In an in-depth study, we explore whether social media discussions from the general public on Bitcoin have predictive power for extreme price movements. A dataset of 5,000 tweets per day containing the keyword `Bitcoin' was collected from 2015 to 2021. This dataset, called PreBit, is made available online. In our hybrid model, we use sentence-level FinBERT embeddings, pretrained on financial lexicons, so as to capture the full contents of the tweets and feed it to the model in an understandable way. By combining these embeddings with a Convolutional Neural Network, we built a predictive model for significant market movements. The final multimodal ensemble model includes this NLP model together with a model based on candlestick data, technical indicators and correlated asset prices. In an ablation study, we explore the contribution of the individual modalities. Finally, we propose and backtest a trading strategy based on the predictions of our models with varying prediction threshold and show that it can used to build a profitable trading strategy with a reduced risk over a `hold' or moving average strategy.