mlsanityData to 7 Oct 2026

Evaluating Financial Intelligence in Large Language Models: Benchmarking SuperInvesting AI with LLM Engines

arXiv 2603.08704 ↗First posted 9 Mar 2026cs.AIcited by 1

On the map: LLM reasoning → Reasoning for agents, planning and code

Abstract (arXiv)

Large language models are increasingly used for financial analysis and investment research, yet systematic evaluation of their financial reasoning capabilities remains limited. In this work, we introduce the AI Financial Intelligence Benchmark (AFIB), a multi-dimensional evaluation framework designed to assess financial analysis capabilities across five dimensions: factual accuracy, analytical completeness, data recency, model consistency, and failure patterns. We evaluate five AI systems: GPT, Gemini, Perplexity, Claude, and SuperInvesting, using a dataset of 95+ structured financial analysis questions derived from real-world equity research tasks. The results reveal substantial differences in performance across models. Within this benchmark setting, SuperInvesting achieves the highest aggregate performance, with an average factual accuracy score of 8.96/10 and the highest completeness score of 56.65/70, while also demonstrating the lowest hallucination rate among evaluated systems. Retrieval-oriented systems such as Perplexity perform strongly on data recency tasks due to live information access but exhibit weaker analytical synthesis and consistency. Overall, the results highlight that financial intelligence in large language models is inherently multi-dimensional, and systems that combine structured financial data access with analytical reasoning capabilities provide the most reliable performance for complex investment research workflows.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 28 of its 34 references that are in the map. First posted on arXiv 9 Mar 2026.

Builds on

28 of 34 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answering7 Oct

Other references · 27

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  6. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  7. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  8. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  9. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,151
  10. 2023BloombergGPT: A Large Language Model for Financecited by 923
  11. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
  12. 2024Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluatorscited by 865
15 more
  1. 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 552
  2. 2024From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipelinecited by 484
  3. 2019FinBERT: Financial Sentiment Analysis with Pre-trained Language Modelscited by 378
  4. 2024TrustLLM: Trustworthiness in Large Language Modelscited by 280
  5. 2023FinanceBench: A New Benchmark for Financial Question Answeringcited by 261
  6. 2022Predictability and Surprise in Large Generative Modelscited by 209
  7. 2024FinBen: A Holistic Financial Benchmark for Large Language Modelscited by 140
  8. 2024A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challengescited by 86
  9. 2023FinGPT: Democratizing Internet-scale Data for Financial Large Language Modelscited by 65
  10. 2024INVESTORBENCH: A Benchmark for Financial Decision-Making Tasks with LLM-based Agentcited by 62
  11. 2025FinanceQA: A Benchmark for Evaluating Financial Analysis Capabilities of Large Language Modelscited by 18
  12. 2025When Agents Trade: Live Multi-Market Trading Benchmark for LLM Agentscited by 16
  13. 2025FinAuditing: A Financial Taxonomy-Structured Multi-Document Benchmark for Evaluating LLMscited by 14
  14. 2025FAITH: A Framework for Assessing Intrinsic Tabular Hallucinations in Financecited by 8
  15. 2026Fin-RATE: A Real-world Financial Analytics and Tracking Evaluation Benchmark for LLMs on SEC Filingscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026AISafetyBenchExplorer: A Metric-Aware Catalogue of AI Safety Benchmarks Reveals Fragmented Measurement and Weak Benchmark Governancecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026QuantEval: A Benchmark for Financial Quantitative Tasks in Large Language Models
  2. 2026Are the Financial Reasoning from LLMs Credible? A Real World Test over Long-Horizon Statements
  3. 2025FinEval-KR: A Financial Domain Evaluation Framework for Large Language Models' Knowledge and Reasoning
  4. 2025Agentar-Fin-R1: Enhancing Financial Intelligence through Domain Expertise, Training Efficiency, and Advanced Reasoning
  5. 2025Towards Competent AI for Fundamental Analysis in Finance: A Benchmark Dataset and Evaluation
  6. 2023FinEval: A Chinese Financial Domain Knowledge Evaluation Benchmark for Large Language Models
  7. 2023Beyond Classification: Financial Reasoning in State-of-the-Art Language Models
  8. 2024FinTral: A Family of GPT-4 Level Multimodal Financial Large Language Models