# Fin-Bias: Comprehensive Evaluation for LLM Decision-Making under human bias in Finance Domain

arXiv: https://arxiv.org/abs/2605.09106 · HTML: https://mlsanity.com/paper/2605.09106/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 43 references that are in the map.
- First posted on arXiv 9 May 2026.

## Builds on (most cited first, 13 of 13)

- [DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model](https://mlsanity.com/paper/2405.04434/) (2024)
- [FinQA: A Dataset of Numerical Reasoning over Financial Data](https://mlsanity.com/paper/2109.00122/) (2021)
- [Good Debt or Bad Debt: Detecting Semantic Orientations in Economic Texts](https://mlsanity.com/paper/1307.5336/) (2013)
- [ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answering](https://mlsanity.com/paper/2210.03849/) (2022)
- [TradingAgents: Multi-Agents LLM Financial Trading Framework](https://mlsanity.com/paper/2412.20138/) (2024)
- [FinMem: A Performance-Enhanced LLM Trading Agent with Layered Memory and Character Design](https://mlsanity.com/paper/2311.13743/) (2023)
- [Reducing Sentiment Bias in Language Models via Counterfactual Evaluation](https://mlsanity.com/paper/1911.03064/) (2019)
- [FinGPT: Instruction Tuning Benchmark for Open-Source Large Language Models in Financial Datasets](https://mlsanity.com/paper/2310.04793/) (2023)
- [Decision-Making Behavior Evaluation Framework for LLMs under Uncertain Context](https://mlsanity.com/paper/2406.05972/) (2024)
- [FinanceQA: A Benchmark for Evaluating Financial Analysis Capabilities of Large Language Models](https://mlsanity.com/paper/2501.18062/) (2025)
- [Mono-InternVL-1.5: Towards Cheaper and Faster Monolithic Multimodal Large Language Models](https://mlsanity.com/paper/2507.12566/) (2025)
- [GPT-Signal: Generative AI for Semi-automated Feature Engineering in the Alpha Research Process](https://mlsanity.com/paper/2410.18448/) (2024)
- [Out of the Memory Barrier: A Highly Memory Efficient Training System for LLMs with Million-Token Contexts](https://mlsanity.com/paper/2602.02108/) (2026)

## Built on by (most cited first, 1 of 1)

- [When Text and Numbers Disagree: Evidence Arbitration in Large Language Models](https://mlsanity.com/paper/2608.20116/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are increasingly deployed in financial contexts, raising critical concerns about reliability, alignment, and susceptibility to adversarial manipulation. While prior finance-related benchmarks assess LLMs' capabilities in stock trading, they are often restricted to small sample and fail to demonstrate LLM susceptibility to context with potential human bias. We introduce Fin-Bias (financial herding under long and uncertain financial context), a benchmark for evaluating LLM investment decision-making when faced with uncertainty and possible human-biased opinions. Fin-Bias includes 8868 long firm-specific analyst reports, including firm aspects summarized and analyzed by sophisticated analysts with investment ratings (Bullish/Neutral/Bearish) spanning from various industries. We present large language models with firm analyst reports with/without analyst investment ratings and even with 'fake' rating, to get investment ratings generated by LLMs. Our results reveal that LLMs tend to herd the explicit bias in context. We also develop a method to detect potential human opinions, which can encourage LLMs to think independently, some models even exceed human performance in predicting future stock return.