# FinGPT: Democratizing Internet-scale Data for Financial Large Language Models

arXiv: https://arxiv.org/abs/2307.10485 · HTML: https://mlsanity.com/paper/2307.10485/

## Key facts

- Cited by 67 papers in the map (data through 14 Oct 2026).
- Builds on 52 of its 92 references that are in the map.
- First posted on arXiv 19 Jul 2023.

## Builds on (most cited first, 30 of 52)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models](https://mlsanity.com/paper/2106.10199/) (2021)
- [GLM: General Language Model Pretraining with Autoregressive Blank Infilling](https://mlsanity.com/paper/2103.10360/) (2021)
- [BloombergGPT: A Large Language Model for Finance](https://mlsanity.com/paper/2303.17564/) (2023)
- [Bag of Tricks for Efficient Text Classification](https://mlsanity.com/paper/1607.01759/) (2016)
- [AdapterFusion: Non-Destructive Task Composition for Transfer Learning](https://mlsanity.com/paper/2005.00247/) (2020)
- [Galactica: A Large Language Model for Science](https://mlsanity.com/paper/2211.09085/) (2022)
- [GPT Understands, Too](https://mlsanity.com/paper/2103.10385/) (2021)
- [GPT-NeoX-20B: An Open-Source Autoregressive Language Model](https://mlsanity.com/paper/2204.06745/) (2022)
- [CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data](https://mlsanity.com/paper/1911.00359/) (2019)

## Built on by (most cited first, 30 of 67)

- [FinCon: A Synthesized LLM Multi-Agent System with Conceptual Verbal Reinforcement for Enhanced Financial Decision Making](https://mlsanity.com/paper/2407.06567/) (2024)
- [Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlook](https://mlsanity.com/paper/2310.10196/) (2023)
- [Revolutionizing Finance with LLMs: An Overview of Applications and Insights](https://mlsanity.com/paper/2401.11641/) (2024)
- [A Survey on LoRA of Large Language Models](https://mlsanity.com/paper/2407.11046/) (2024)
- [FLEX: Continuous Agent Evolution via Forward Learning from Experience](https://mlsanity.com/paper/2511.06449/) (2025)
- [Differentially Private Low-Rank Adaptation of Large Language Model Using Federated Learning](https://mlsanity.com/paper/2312.17493/) (2023)
- [TelecomGPT: A Framework to Build Telecom-Specfic Large Language Models](https://mlsanity.com/paper/2407.09424/) (2024)
- [MarS: a Financial Market Simulation Engine Powered by Generative Foundation Model](https://mlsanity.com/paper/2409.07486/) (2024)
- [Benchmark Test-Time Scaling of General LLM Agents](https://mlsanity.com/paper/2602.18998/) (2026)
- [FinLlama: Financial Sentiment Classification for Algorithmic Trading Applications](https://mlsanity.com/paper/2403.12285/) (2024)
- [PrefillOnly: An Inference Engine for Prefill-only Workloads in Large Language Model Applications](https://mlsanity.com/paper/2505.07203/) (2025)
- [Graph Meets LLMs: Towards Large Graph Models](https://mlsanity.com/paper/2308.14522/) (2023)
- [LLM-Powered Multi-Agent System for Automated Crypto Portfolio Management](https://mlsanity.com/paper/2501.00826/) (2025)
- [User Modeling in the Era of Large Language Models: Current Research and Future Directions](https://mlsanity.com/paper/2312.11518/) (2023)
- [PanGu-π: Enhancing Language Model Architectures via Nonlinearity Compensation](https://mlsanity.com/paper/2312.17276/) (2023)
- [Memory Decoder: A Pretrained, Plug-and-Play Memory for Large Language Models](https://mlsanity.com/paper/2508.09874/) (2025)
- [FinChain: A Symbolic Benchmark for Verifiable Chain-of-Thought Financial Reasoning](https://mlsanity.com/paper/2506.02515/) (2025)
- [GenAINet: Enabling Wireless Collective Intelligence via Knowledge Transfer and Reasoning](https://mlsanity.com/paper/2402.16631/) (2024)
- [Learning Facts at Scale with Active Reading](https://mlsanity.com/paper/2508.09494/) (2025)
- [Needle Threading: Can LLMs Follow Threads through Near-Million-Scale Haystacks?](https://mlsanity.com/paper/2411.05000/) (2024)
- [The Future of Open Human Feedback](https://mlsanity.com/paper/2408.16961/) (2024)
- [GreedLlama: Performance of Financial Value-Aligned Large Language Models in Moral Reasoning](https://mlsanity.com/paper/2404.02934/) (2024)
- [PatentGPT: A Large Language Model for Intellectual Property](https://mlsanity.com/paper/2404.18255/) (2024)
- [TRIDENT: Benchmarking LLM Safety in Finance, Medicine, and Law](https://mlsanity.com/paper/2507.21134/) (2025)
- [An Empirical Categorization of Prompting Techniques for Large Language Models: A Practitioner's Guide](https://mlsanity.com/paper/2402.14837/) (2024)
- [GPT-4 as Evaluator: Evaluating Large Language Models on Pest Management in Agriculture](https://mlsanity.com/paper/2403.11858/) (2024)
- [Multi-Document Financial Question Answering using LLMs](https://mlsanity.com/paper/2411.07264/) (2024)
- [From ASR to ASP: Evaluating Prompt Attack Vulnerabilities Against Open-Source LLMs](https://mlsanity.com/paper/2505.14368/) (2025)
- [Benchmarking Multi-Agent LLM Architectures for Financial Document Processing: A Comparative Study of Orchestration Patterns, Cost-Accuracy Tradeoffs and Production Scaling Strategies](https://mlsanity.com/paper/2603.22651/) (2026)
- [Evolution of meta's llama models and parameter-efficient fine-tuning of large language models: a survey](https://mlsanity.com/paper/2510.12178/) (2025)

## Abstract (arXiv)

Large language models (LLMs) have demonstrated remarkable proficiency in understanding and generating human-like texts, which may potentially revolutionize the finance industry. However, existing LLMs often fall short in the financial field, which is mainly attributed to the disparities between general text data and financial text data. Unfortunately, there is only a limited number of financial text datasets available, and BloombergGPT, the first financial LLM (FinLLM), is close-sourced (only the training logs were released). In light of this, we aim to democratize Internet-scale financial data for LLMs, which is an open challenge due to diverse data sources, low signal-to-noise ratio, and high time-validity. To address the challenges, we introduce an open-sourced and data-centric framework, Financial Generative Pre-trained Transformer (FinGPT), that automates the collection and curation of real-time financial data from 34 diverse sources on the Internet, providing researchers and practitioners with accessible and transparent resources to develop their FinLLMs. Additionally, we propose a simple yet effective strategy for fine-tuning FinLLM using the inherent feedback from the market, dubbed Reinforcement Learning with Stock Prices (RLSP). We also adopt the Low-rank Adaptation (LoRA, QLoRA) method that enables users to customize their own FinLLMs from general-purpose LLMs at a low cost. Finally, we showcase several FinGPT applications, including robo-advisor, sentiment analysis for algorithmic trading, and low-code development. FinGPT aims to democratize FinLLMs, stimulate innovation, and unlock new opportunities in open finance. The codes have been open-sourced.