# Improved LLM Agents for Financial Document Question Answering

arXiv: https://arxiv.org/abs/2506.08726 · HTML: https://mlsanity.com/paper/2506.08726/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Financial trading and forecasting agents, one of 16 directions in LLM agents and tool use.
- Builds on 13 of its 25 references that are in the map.
- First posted on arXiv 10 Jun 2025.

## Placed in

- LLM agents and tool use / [Financial trading and forecasting agents](https://mlsanity.com/agents/financial-trading-and-forecasting-agents/)

## Builds on (most cited first, 13 of 13)

- [Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks](https://mlsanity.com/paper/2211.12588/) (2022)
- [Large Language Models Cannot Self-Correct Reasoning Yet](https://mlsanity.com/paper/2310.01798/) (2023)
- [FinQA: A Dataset of Numerical Reasoning over Financial Data](https://mlsanity.com/paper/2109.00122/) (2021)
- [TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance](https://mlsanity.com/paper/2105.07624/) (2021)
- [Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understanding](https://mlsanity.com/paper/2401.04398/) (2024)
- [Number Cookbook: Number Understanding of Language Models and How to Improve It](https://mlsanity.com/paper/2411.03766/) (2024)
- [H-STAR: LLM-driven Hybrid SQL-Text Adaptive Reasoning on Tables](https://mlsanity.com/paper/2407.05952/) (2024)
- [Tree-of-Table: Unleashing the Power of LLMs for Enhanced Large-Scale Table Understanding](https://mlsanity.com/paper/2411.08516/) (2024)
- [TAT-LLM: A Specialized Language Model for Discrete Reasoning over Tabular and Textual Data](https://mlsanity.com/paper/2401.13223/) (2024)
- [API-Assisted Code Generation for Question Answering on Varied Table Structures](https://mlsanity.com/paper/2310.14687/) (2023)
- [Enhancing Financial Question Answering with a Multi-Agent Reflection Framework](https://mlsanity.com/paper/2410.21741/) (2024)
- [SynTQA: Synergistic Table-based Question Answering via Mixture of Text-to-SQL and E2E TQA](https://mlsanity.com/paper/2409.16682/) (2024)
- [LLM Reasoning Engine: Specialized Training for Enhanced Mathematical Reasoning](https://mlsanity.com/paper/2412.20227/) (2024)

## Built on by (most cited first, 5 of 5)

- [Does Using Counterfactual Help LLMs Explain Textual Importance in Classification?](https://mlsanity.com/paper/2510.04031/) (2025)
- [Reasoning With a Star: A Heliophysics Dataset and Benchmark for Agentic Scientific Reasoning](https://mlsanity.com/paper/2511.20694/) (2025)
- [Improved Evidence Extraction and Metrics for Document Inconsistency Detection with LLMs](https://mlsanity.com/paper/2601.02627/) (2026)
- [MoCA-Agent: A Market-of-Claims Code Agent for Financial and Numerical Reasoning](https://mlsanity.com/paper/2606.11537/) (2026)
- [Same-Number Citation Swaps: Stress-Testing Jev as a Financial Evidence Judge](https://mlsanity.com/paper/2610.08675/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have shown impressive capabilities on numerous natural language processing tasks. However, LLMs still struggle with numerical question answering for financial documents that include tabular and textual data. Recent works have showed the effectiveness of critic agents (i.e., self-correction) for this task given oracle labels. Building upon this framework, this paper examines the effectiveness of the traditional critic agent when oracle labels are not available, and show, through experiments, that this critic agent's performance deteriorates in this scenario. With this in mind, we present an improved critic agent, along with the calculator agent which outperforms the previous state-of-the-art approach (program-of-thought) and is safer. Furthermore, we investigate how our agents interact with each other, and how this interaction affects their performance.