mlsanityData to 7 Oct 2026

BloombergGPT: A Large Language Model for Finance

arXiv 2303.17564 ↗First posted 30 Mar 2023cs.LG cs.AI cs.CLcited by 923

Not on the map. Why?

Cited by 102 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The use of NLP in the realm of financial technology is broad and complex, with applications ranging from sentiment analysis and named entity recognition to question answering. Large Language Models (LLMs) have been shown to be effective on a variety of tasks; however, no LLM specialized for the financial domain has been reported in literature. In this work, we present BloombergGPT, a 50 billion parameter language model that is trained on a wide range of financial data. We construct a 363 billion token dataset based on Bloomberg's extensive data sources, perhaps the largest domain-specific dataset yet, augmented with 345 billion tokens from general purpose datasets. We validate BloombergGPT on standard LLM benchmarks, open financial benchmarks, and a suite of internal benchmarks that most accurately reflect our intended usage. Our mixed dataset training leads to a model that outperforms existing models on financial tasks by significant margins without sacrificing performance on general LLM benchmarks. Additionally, we explain our modeling choices, training process, and evaluation methodology. We release Training Chronicles (Appendix C) detailing our experience in training BloombergGPT.

Key facts. Cited by 923 papers in the map (data through 7 Oct 2026). Builds on 99 of its 138 references that are in the map. First posted on arXiv 30 Mar 2023.

Builds on

99 of 138 references.

References · 99

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  7. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  8. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  9. 2020Scaling Laws for Neural Language Modelscited by 6,411
  10. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  11. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  12. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
38 more
  1. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  2. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  3. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  4. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  5. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  6. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  7. 2022Emergent Abilities of Large Language Modelscited by 2,483
  8. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  9. 2020Extracting Training Data from Large Language Modelscited by 2,144
  10. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  11. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  12. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  13. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  14. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  15. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  16. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  17. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  18. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  19. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  20. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
  21. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  22. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,404
  23. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  24. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  25. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  26. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,085
  27. 2021Ethical and social risks of harm from Language Modelscited by 957
  28. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
  29. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 851
  30. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  31. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  32. 2016Training Deep Nets with Sublinear Memory Costcited by 825
  33. 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 798
  34. 2022Quantifying Memorization Across Neural Language Modelscited by 755
  35. 2022Galactica: A Large Language Model for Sciencecited by 736
  36. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  37. 2021Deduplicating Training Data Makes Language Models Bettercited by 705
  38. 2019Compressive Transformers for Long-Range Sequence Modellingcited by 699

50 most cited of 99 All 99 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 168; 2024: 304; 2025: 249; 2026: 202.

923 papers.

Sort:

By area:

  1. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  2. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  3. 2023A Comprehensive Overview of Large Language Modelscited by 639
  4. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 434
  5. 2023Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyondcited by 409
  6. 2023Llemma: An Open Language Model For Mathematicscited by 385
  7. 2023LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Compositioncited by 359
  8. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  9. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 276
  10. 2023VerilogEval: Evaluating Large Language Models for Verilog Code Generationcited by 246
  11. 2023FinGPT: Open-Source Financial Large Language Modelscited by 244
  12. 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 228
38 more
  1. 2023Large Language Models in Finance: A Surveycited by 223
  2. 2023PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Financecited by 217
  3. 2023MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Usecited by 209
  4. 2024Tower: An Open Multilingual Large Language Model for Translation-Related Taskscited by 205
  5. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
  6. 2023Evaluating Large Language Models: A Comprehensive Surveycited by 200
  7. 2023Recommender Systems in the Era of Large Language Models (LLMs)cited by 188
  8. 2024Large Language Models for Education: A Survey and Outlookcited by 185
  9. 2023Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMscited by 175
  10. 2023ChipNeMo: Domain-Adapted LLMs for Chip Designcited by 168
  11. 2023FinMem: A Performance-Enhanced LLM Trading Agent with Layered Memory and Character Designcited by 163
  12. 2023How Can Recommender Systems Benefit from Large Language Models: A Surveycited by 141
  13. 2024FinBen: A Holistic Financial Benchmark for Large Language Modelscited by 140
  14. 2024A Multimodal Foundation Agent for Financial Trading: Tool-Augmented, Diversified, and Generalistcited by 137
  15. 2023Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Surveycited by 134
  16. 2024LHRS-Bot: Empowering Remote Sensing with VGI-Enhanced Large Multimodal Language Modelcited by 133
  17. 2025LLM Post-Training: A Deep Dive into Reasoning Large Language Modelscited by 129
  18. 2023Can ChatGPT Forecast Stock Price Movements? Return Predictability and Large Language Modelscited by 125
  19. 2024OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learningcited by 124
  20. 2024GTBench: Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluationscited by 118
  21. 2024Machine Unlearning of Pre-trained Large Language Modelscited by 118
  22. 2023Evaluating and Mitigating Discrimination in Language Model Decisionscited by 117
  23. 2024Large Language Models for Time Series: A Surveycited by 113
  24. 2023Bayesian Low-rank Adaptation for Large Language Modelscited by 112
  25. 2023M3Exam: A Multilingual, Multimodal, Multilevel Benchmark for Examining Large Language Modelscited by 109
  26. 2023Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlookcited by 109
  27. 2024LLaMA Pro: Progressive LLaMA with Block Expansioncited by 99
  28. 2023Enhancing Uncertainty-Based Hallucination Detection with Stronger Focuscited by 98
  29. 2023XuanYuan 2.0: A Large Chinese Financial Chat Model with Hundreds of Billions Parameterscited by 92
  30. 2024ChatScene: Knowledge-Enabled Safety-Critical Scenario Generation for Autonomous Vehiclescited by 92
  31. 2024HybridRAG: Integrating Knowledge Graphs and Vector Retrieval Augmented Generation for Efficient Information Extractioncited by 92
  32. 2023A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validationcited by 91
  33. 2024Self-Distillation Bridges Distribution Gap in Language Model Fine-Tuningcited by 90
  34. 2024RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMscited by 89
  35. 2024Revolutionizing Finance with LLMs: An Overview of Applications and Insightscited by 88
  36. 2024A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challengescited by 86
  37. 2024Large Language Model (LLM) for Telecommunications: A Comprehensive Survey on Principles, Key Techniques, and Opportunitiescited by 83
  38. 2025Fin-R1: A Large Language Model for Financial Reasoning through Reinforcement Learningcited by 82

50 most cited of 923 All 923 (JSON)