mlsanityData to 7 Oct 2026

Are ChatGPT and GPT-4 General-Purpose Solvers for Financial Text Analytics? A Study on Several Typical Tasks

arXiv 2305.05862 ↗First posted 10 May 2023cs.CL cs.AIcited by 59

Not on the map. Why?

Abstract (arXiv)

The most recent large language models(LLMs) such as ChatGPT and GPT-4 have shown exceptional capabilities of generalist models, achieving state-of-the-art performance on a wide range of NLP tasks with little or no adaptation. How effective are such models in the financial domain? Understanding this basic question would have a significant impact on many downstream financial analytical tasks. In this paper, we conduct an empirical study and provide experimental evidences of their performance on a wide variety of financial text analytical problems, using eight benchmark datasets from five categories of tasks. We report both the strengths and limitations of the current models by comparing them to the state-of-the-art fine-tuned approaches and the recently released domain-specific pretrained models. We hope our study can help understand the capability of the existing models in the financial domain and facilitate further improvements.

Key facts. Cited by 59 papers in the map (data through 7 Oct 2026). Builds on 21 of its 35 references that are in the map. First posted on arXiv 10 May 2023.

Builds on

21 of 35 references.

References · 21

  1. 2017Deep reinforcement learning from human preferencescited by 3,846
  2. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  3. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  4. 2023BloombergGPT: A Large Language Model for Financecited by 923
  5. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  6. 2022Galactica: A Large Language Model for Sciencecited by 736
  7. 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 552
  8. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 531
  9. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 486
  10. 2019FinBERT: Financial Sentiment Analysis with Pre-trained Language Modelscited by 378
  11. 2020LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attentioncited by 326
  12. 2013Good Debt or Bad Debt: Detecting Semantic Orientations in Economic Textscited by 310
9 more
  1. 2023Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4cited by 202
  2. 2022ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answeringcited by 194
  3. 2020Global Table Extractor (GTE): A Framework for Joint Table Identification and Cell Structure Recognition Using Visual Contextcited by 146
  4. 2022WHEN FLUE MEETS FLANG: Benchmarks and Large Pre-trained Language Model for Financial Domaincited by 117
  5. 2022What Language Model to Train if You Have One Million GPU Hours?cited by 92
  6. 2020Impact of News on the Commodity Market: Dataset and Resultscited by 80
  7. 2023How Secure is Code Generated by ChatGPT?cited by 77
  8. 2023ChatGPT versus Traditional Question Answering for Knowledge Graphs: Current Status and Future Directions Towards Knowledge Graph Chatbotscited by 47
  9. 2022Learning Better Intent Representations for Financial Open Intent Classificationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 25; 2025: 24; 2026: 6.

59 papers.

Sort:

By area:

  1. 2023Benchmarking Large Language Models in Retrieval-Augmented Generationcited by 307
  2. 2024A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challengescited by 86
  3. 2024Simple Is Effective: The Roles of Graphs and Large Language Models in Knowledge-Graph-Based Retrieval-Augmented Generationcited by 78
  4. 2024A Survey of Large Language Models in Finance (FinLLMs)cited by 77
  5. 2024CRUD-RAG: A Comprehensive Chinese Benchmark for Retrieval-Augmented Generation of Large Language Modelscited by 50
  6. 2024LongRAG: A Dual-Perspective Retrieval-Augmented Generation Paradigm for Long-Context Question Answeringcited by 49
  7. 2024Can Large Language Models Beat Wall Street? Unveiling the Potential of AI in Stock Selectioncited by 29
  8. 2025Entropy-Gated Branching for Efficient Test-Time Reasoningcited by 21
  9. 2025MoC: Mixtures of Text Chunking Learners for Retrieval-Augmented Generation Systemcited by 19
  10. 2023Navigating Prompt Complexity for Zero-Shot Classification: A Study of Large Language Models in Computational Social Sciencecited by 18
  11. 2024Large Language Models as Financial Data Annotators: A Study on Effectiveness and Efficiencycited by 17
  12. 2024Meta-Chunking: Learning Text Segmentation and Semantic Completion via Logical Perceptioncited by 17
38 more
  1. 2024SMARTCAL: An Approach to Self-Aware Tool-Use Evaluation and Calibrationcited by 13
  2. 2024LLMs with Industrial Lens: Deciphering the Challenges and Prospects -- A Surveycited by 12
  3. 2024Evaluating AI for Law: Bridging the Gap with Open-Source Solutionscited by 11
  4. 2025HiChunk: Evaluating and Enhancing Retrieval-Augmented Generation with Hierarchical Chunkingcited by 9
  5. 2024A Comparative Analysis of Instruction Fine-Tuning LLMs for Financial Text Classificationcited by 8
  6. 2025RAGRouter: Learning to Route Queries to Multiple Retrieval-Augmented Language Modelscited by 8
  7. 2025Financial Named Entity Recognition: How Far Can LLM Go?cited by 7
  8. 2025Evaluating Knowledge Graph Based Retrieval Augmented Generation Methods under Knowledge Incompletenesscited by 7
  9. 2024Opportunities and Challenges of Generative-AI in Financecited by 6
  10. 2024Zero-Shot Topic Classification of Column Headers: Leveraging LLMs for Metadata Enrichmentcited by 5
  11. 2023Chinese Fine-Grained Financial Sentiment Analysis with Large Language Modelscited by 4
  12. 2025Bridging Language Models and Financial Analysiscited by 4
  13. 2025FG-RAG: Enhancing Query-Focused Summarization with Context-Aware Fine-Grained Graph RAGcited by 4
  14. 2024ABC Align: Large Language Model Alignment for Safety & Accuracycited by 3
  15. 2024No Size Fits All: The Perils and Pitfalls of Leveraging LLMs Vary with Company Sizecited by 3
  16. 2024Quantifying Qualitative Insights: Leveraging LLMs to Market Predictcited by 3
  17. 2025Extract, Match, and Score: An Evaluation Paradigm for Long Question-context-answer Triplets in Financial Analysiscited by 3
  18. 2025FinNLI: Novel Dataset for Multi-Genre Financial Natural Language Inference Benchmarkingcited by 3
  19. 2025Masking in Multi-hop QA: An Analysis of How Language Models Perform with Context Permutationcited by 3
  20. 2025Weak-to-Strong GraphRAG: Aligning Weak Retrievers with Large Language Models for Graph-based Retrieval Augmented Generationcited by 3
  21. 2025Evaluating Large Language Models for Financial Reasoning: A CFA-Based Benchmark Studycited by 3
  22. 2024Economy Watchers Survey Provides Datasets and Tasks for Japanese Financial Domaincited by 2
  23. 2024UCFE: A User-Centric Financial Expertise Benchmark for Large Language Modelscited by 2
  24. 2023Evolution of ESG-focused DLT Research: An NLP Analysis of the Literaturecited by 1
  25. 2024Boardwalk Empire: How Generative AI is Revolutionizing Economic Paradigmscited by 1
  26. 2024Can General-Purpose Large Language Models Generalize to English-Thai Machine Translation ?cited by 1
  27. 2024Refined and Segmented Price Sentiment Indices from Survey Commentscited by 1
  28. 2025Exploring the Reliability of Self-explanation and its Relationship with Classification in Language Model-driven Financial Analysiscited by 1
  29. 2025Rethinking the Understanding Ability across LLMs through Mutual Informationcited by 1
  30. 2025AI Analyst: Framework and Comprehensive Evaluation of Large Language Models for Financial Time Series Report Generationcited by 1
  31. 2025Are LLMs Enough for Hyperpartisan, Fake, Polarized and Harmful Content Detection? Evaluating In-Context Learning vs. Fine-Tuningcited by 1
  32. 2025MoM: Mixtures of Scenario-Aware Document Memories for Retrieval-Augmented Generation Systemscited by 1
  33. 2024Projective Methods for Mitigating Gender Bias in Pre-trained Language Modelscited by 0
  34. 2024A Survey on the Real Power of ChatGPTcited by 0
  35. 2024Column Vocabulary Association (CVA): semantic interpretation of dataless tablescited by 0
  36. 2025A Transformer-based Autoregressive Decoder Architecture for Hierarchical Text Classificationcited by 0
  37. 2025CRAB: A Benchmark for Evaluating Curation of Retrieval-Augmented LLMs in Biomedicinecited by 0
  38. 2025A Variational Approach for Mitigating Entity Bias in Relation Extractioncited by 0

50 most cited of 59 All 59 (JSON)