# Leveraging Vision-Language Models for Granular Market Change Prediction

arXiv: https://arxiv.org/abs/2301.10166 · HTML: https://mlsanity.com/paper/2301.10166/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 25 references that are in the map.
- First posted on arXiv 17 Jan 2023.

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [FinBERT: Financial Sentiment Analysis with Pre-trained Language Models](https://mlsanity.com/paper/1908.10063/) (2019)
- [Not All Relevance Scores are Equal: Efficient Uncertainty and Calibration Modeling for Deep Retrieval Models](https://mlsanity.com/paper/2105.04651/) (2021)
- [Volatility Prediction using Financial Disclosures Sentiments with Word Embedding-based IR Models](https://mlsanity.com/paper/1702.01978/) (2017)
- [Text Mining of Stocktwits Data for Predicting Stock Prices](https://mlsanity.com/paper/2103.16388/) (2021)
- [Portfolio Optimization with 2D Relative-Attentional Gated Transformer](https://mlsanity.com/paper/2101.03138/) (2020)
- [Exploiting Investors Social Network for Stock Prediction in China's Market](https://mlsanity.com/paper/1801.00597/) (2018)

## Built on by (most cited first, 22 of 22)

- [Foundation Models for Time Series Analysis: A Tutorial and Survey](https://mlsanity.com/paper/2403.14735/) (2024)
- [Are Language Models Actually Useful for Time Series Forecasting?](https://mlsanity.com/paper/2406.16964/) (2024)
- [Large Language Models for Time Series: A Survey](https://mlsanity.com/paper/2402.01801/) (2024)
- [A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challenges](https://mlsanity.com/paper/2406.11903/) (2024)
- [VisionTS: Visual Masked Autoencoders Are Free-Lunch Zero-Shot Time Series Forecasters](https://mlsanity.com/paper/2408.17253/) (2024)
- [Can LLMs Understand Time Series Anomalies?](https://mlsanity.com/paper/2410.05440/) (2024)
- [Toward Foundation Model for Multivariate Wearable Sensing of Physiological Signals](https://mlsanity.com/paper/2412.09758/) (2024)
- [Harnessing Vision Models for Time Series Analysis: A Survey](https://mlsanity.com/paper/2502.08869/) (2025)
- [LanP: Rethinking the Impact of Language Priors in Large Vision-Language Models](https://mlsanity.com/paper/2502.12359/) (2025)
- [VisionTS++: Cross-Modal Time Series Foundation Model with Continual Pre-trained Vision Backbones](https://mlsanity.com/paper/2508.04379/) (2025)
- [Unraveling Spatio-Temporal Foundation Models via the Pipeline Lens: A Comprehensive Review](https://mlsanity.com/paper/2506.01364/) (2025)
- [Multi-Modal View Enhanced Large Vision Models for Long-Term Time Series Forecasting](https://mlsanity.com/paper/2505.24003/) (2025)
- [From Images to Signals: Are Large Vision Models Useful for Time Series Analysis?](https://mlsanity.com/paper/2505.24030/) (2025)
- [Learning Transferable Sensor Models via Language-Informed Pretraining](https://mlsanity.com/paper/2603.11950/) (2026)
- [Methods for Acquiring and Incorporating Knowledge into Stock Price Prediction: A Survey](https://mlsanity.com/paper/2308.04947/) (2023)
- [SURGE: On the Potential of Large Language Models as General-Purpose Surrogate Code Executors](https://mlsanity.com/paper/2502.11167/) (2025)
- [Enhancing Traffic Prediction with Textual Data Using Large Language Models](https://mlsanity.com/paper/2405.06719/) (2024)
- [SensorBench: Benchmarking LLMs in Coding-Based Sensor Processing](https://mlsanity.com/paper/2410.10741/) (2024)
- [UCFE: A User-Centric Financial Expertise Benchmark for Large Language Models](https://mlsanity.com/paper/2410.14059/) (2024)
- [Temporal-Visual Semantic Alignment: A Unified Architecture for Transferring Spatial Priors from Vision Models to Zero-Shot Temporal Tasks](https://mlsanity.com/paper/2511.19856/) (2025)
- [Neural Operators Can Discover Functional Clusters](https://mlsanity.com/paper/2602.23528/) (2026)
- [Distribution-Aware Diffusion-LLM for Robust Ultra-Long-Term Time Series Forecasting](https://mlsanity.com/paper/2606.23391/) (2026)

## Abstract (arXiv)

Predicting future direction of stock markets using the historical data has been a fundamental component in financial forecasting. This historical data contains the information of a stock in each specific time span, such as the opening, closing, lowest, and highest price. Leveraging this data, the future direction of the market is commonly predicted using various time-series models such as Long-Short Term Memory networks. This work proposes modeling and predicting market movements with a fundamentally new approach, namely by utilizing image and byte-based number representation of the stock data processed with the recently introduced Vision-Language models. We conduct a large set of experiments on the hourly stock data of the German share index and evaluate various architectures on stock price prediction using historical stock data. We conduct a comprehensive evaluation of the results with various metrics to accurately depict the actual performance of various approaches. Our evaluation results show that our novel approach based on representation of stock data as text (bytes) and image significantly outperforms strong deep learning-based baselines.