mlsanityData to 7 Oct 2026

FinVis-GPT: A Multimodal Large Language Model for Financial Chart Analysis

arXiv 2308.01430 ↗First posted 31 Jul 2023cs.CLcited by 27

Not on the map. Why?

Abstract (arXiv)

In this paper, we propose FinVis-GPT, a novel multimodal large language model (LLM) specifically designed for financial chart analysis. By leveraging the power of LLMs and incorporating instruction tuning and multimodal capabilities, FinVis-GPT is capable of interpreting financial charts and providing valuable analysis. To train FinVis-GPT, a financial task oriented dataset was generated for pre-training alignment and instruction tuning, comprising various types of financial charts and their corresponding descriptions. We evaluate the model performance via several case studies due to the time limit, and the promising results demonstrated that FinVis-GPT is superior in various financial chart related tasks, including generating descriptions, answering questions and predicting future market trends, surpassing existing state-of-the-art multimodal LLMs. The proposed FinVis-GPT serves as a pioneering effort in utilizing multimodal LLMs in the finance domain and our generated dataset will be release for public use in the near future to speedup related research.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Builds on 9 of its 10 references that are in the map. First posted on arXiv 31 Jul 2023.

Builds on

9 of 10 references.

References · 9

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023Visual Instruction Tuningcited by 7,792
  3. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  4. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  5. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  6. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,038
  7. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Modelcited by 602
  8. 2023Language Is Not All You Need: Aligning Perception with Language Modelscited by 543
  9. 2023Larger language models do in-context learning differentlycited by 386

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 4; 2025: 13; 2026: 9.

27 papers.

Sort:

By area:

  1. 2024A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challengescited by 86
  2. 2025GuardReasoner-VL: Safeguarding VLMs via Reinforced Reasoningcited by 38
  3. 2025Visual Large Language Models for Generalized and Specialized Applicationscited by 35
  4. 2024FinTral: A Family of GPT-4 Level Multimodal Financial Large Language Modelscited by 32
  5. 2025MLLM-CL: Continual Learning for Multimodal Large Language Modelscited by 21
  6. 2025MCITlib: Multimodal Continual Instruction Tuning Library and Benchmarkcited by 10
  7. 2024No Language is an Island: Unifying Chinese and English in Financial Large Language Models, Instruction Data, and Benchmarkscited by 7
  8. 2026Agentic Trading: When LLM Agents Meet Financial Marketscited by 7
  9. 2023Co-audit: tools to help humans double-check AI-generated contentcited by 5
  10. 2025FinTeam: A Multi-Agent Collaborative Intelligence System for Comprehensive Financial Scenarioscited by 5
  11. 2024Golden Touchstone: A Comprehensive Bilingual Benchmark for Evaluating Financial Large Language Modelscited by 4
  12. 2025Bridging Language Models and Financial Analysiscited by 4
15 more
  1. 2025Advancing Financial Engineering with Foundation Models: Progress, Applications, and Challengescited by 4
  2. 2026Spectral Imbalance Causes Forgetting in Low-Rank Continual Adaptationcited by 4
  3. 2025Insight Miner: A Time Series Analysis Dataset for Cross-Domain Alignment with Natural Languagecited by 3
  4. 2025FinLMM-R1: Enhancing Financial Reasoning in LMM through Scalable Data and Reward Designcited by 2
  5. 2025FediLoRA: Practical Federated Fine-Tuning of Foundation Models Under Missing-Modality Constraintscited by 1
  6. 2026Multimodal Information Fusion for Chart Understanding: A Survey of MLLMs -- Evolution, Limitations, and Cognitive Enhancementcited by 1
  7. 2026Echoes of ownership: Adversarial-guided dual injection for copyright protection in MLLMscited by 1
  8. 2025DBOT: Artificial Intelligence for Systematic Long-Term Investingcited by 0
  9. 2025BlindSight: Harnessing Sparsity for Efficient Vision-Language Modelscited by 0
  10. 2025PyFi: Toward Pyramid-like Financial Image Understanding for VLMs via Adversarial Agentscited by 0
  11. 2026UniFinEval: Towards Unified Evaluation of Financial Multimodal Models across Text, Images and Videoscited by 0
  12. 2026KeepLoRA++: Continual Learning with Layer-Scaled Residual Gradient Adaptationcited by 0
  13. 2026FinSAgent: Corpus-Aligned Multi-Agent RAG Framework for Evidence-Grounded SEC Filing Question Answeringcited by 0
  14. 2026SiGMA: Sign-Guided Merging and Adaptation for Multimodal Continual Instruction Tuningcited by 0
  15. 2026Progressive Multimodal Alignment for Continual Instruction Tuningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Open-FinLLMs: Open Multimodal Large Language Models for Financial Applications
  2. 2024FinBen: A Holistic Financial Benchmark for Large Language Models
  3. 2025FinMME: Benchmark Dataset for Financial Multi-Modal Reasoning Evaluation
  4. 2025FinMR: A Knowledge-Intensive Multimodal Benchmark for Advanced Financial Reasoning
  5. 2025FinMMR: Make Financial Numerical Reasoning More Multimodal, Comprehensive, and Challenging
  6. 2026MM-FinEval: A Multi-Task Multimodal Benchmark for Real-World Financial Forecasting
  7. 2025CFBenchmark-MM: Chinese Financial Assistant Benchmark for Multimodal Large Language Model
  8. 2026Are Large Language Models Reliable Reviewers? A Benchmark for Error Detection in Financial Documents