# A Citation-Grounded Benchmark for Trustworthy Earnings Call Transcript Analysis with Large Language Models

arXiv: https://arxiv.org/abs/2610.00969 · HTML: https://mlsanity.com/paper/2610.00969/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 49 references that are in the map.
- First posted on arXiv 1 Oct 2026.

## Builds on (most cited first, 28 of 28)

- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [WebGPT: Browser-assisted question-answering with human feedback](https://mlsanity.com/paper/2112.09332/) (2021)
- [Enabling Large Language Models to Generate Text with Citations](https://mlsanity.com/paper/2305.14627/) (2023)
- [Evaluating Verifiability in Generative Search Engines](https://mlsanity.com/paper/2304.09848/) (2023)
- [RARR: Researching and Revising What Language Models Say, Using Language Models](https://mlsanity.com/paper/2210.08726/) (2022)
- [L-Eval: Instituting Standardized Evaluation for Long Context Language Models](https://mlsanity.com/paper/2307.11088/) (2023)
- [VERISCORE: Evaluating the factuality of verifiable claims in long-form text generation](https://mlsanity.com/paper/2406.19276/) (2024)
- [ECTSum: A New Benchmark Dataset For Bullet Point Summarization of Long Earnings Call Transcripts](https://mlsanity.com/paper/2210.12467/) (2022)
- [Towards Verifiable Generation: A Benchmark for Knowledge-aware Language Model Attribution](https://mlsanity.com/paper/2310.05634/) (2023)
- [PROXYQA: An Alternative Framework for Evaluating Long-Form Text Generation with Large Language Models](https://mlsanity.com/paper/2401.15042/) (2024)
- [Towards Verifiable Text Generation with Evolving Memory and Self-Reflection](https://mlsanity.com/paper/2312.09075/) (2023)
- [Large Language Models as Foundations for Next-Gen Dense Retrieval: A Comprehensive Empirical Assessment](https://mlsanity.com/paper/2408.12194/) (2024)
- [Modeling financial analysts' decision making via the pragmatics and semantics of earnings calls](https://mlsanity.com/paper/1906.02868/) (2019)
- [Volatility Prediction using Financial Disclosures Sentiments with Word Embedding-based IR Models](https://mlsanity.com/paper/1702.01978/) (2017)
- [Greenback Bears and Fiscal Hawks: Finance is a Jungle and Text Embeddings Must Adapt](https://mlsanity.com/paper/2411.07142/) (2024)
- [GraphCheck: Breaking Long-Term Text Barriers with Extracted Knowledge Graph-Powered Fact-Checking](https://mlsanity.com/paper/2502.16514/) (2025)
- [SubjECTive-QA: Measuring Subjectivity in Earnings Call Transcripts' QA Through Six-Dimensional Feature Analysis](https://mlsanity.com/paper/2410.20651/) (2024)
- [FinTrust: A Comprehensive Benchmark of Trustworthiness Evaluation in Finance Domain](https://mlsanity.com/paper/2510.15232/) (2025)
- [STRUX: An LLM for Decision-Making with Structured Explanations](https://mlsanity.com/paper/2410.12583/) (2024)
- [From Facts to Insights: A Study on the Generation and Evaluation of Analytical Reports for Deciphering Earnings Calls](https://mlsanity.com/paper/2410.01039/) (2024)
- [Causal Inference from Text: Unveiling Interactions between Variables](https://mlsanity.com/paper/2311.05286/) (2023)
- [Quantifying the Impact of Structured Output Format on Large Language Models through Causal Inference](https://mlsanity.com/paper/2509.21791/) (2025)
- [Numerical Claim Detection in Finance: A New Financial Dataset, Weak-Supervision Model, and Market Analysis](https://mlsanity.com/paper/2402.11728/) (2024)
- [Measuring Consistency in Text-based Financial Forecasting Models](https://mlsanity.com/paper/2305.08524/) (2023)
- [Exploring the Impact of Corpus Diversity on Financial Pretrained Language Models](https://mlsanity.com/paper/2310.13312/) (2023)
- [Extract, Match, and Score: An Evaluation Paradigm for Long Question-context-answer Triplets in Financial Analysis](https://mlsanity.com/paper/2503.16575/) (2025)
- [Exploring the Reliability of Self-explanation and its Relationship with Classification in Language Model-driven Financial Analysis](https://mlsanity.com/paper/2503.15985/) (2025)

## Abstract (arXiv)

Large language models (LLMs) have been increasingly used for financial document analysis, including earnings call transcripts (ECTs). Beyond generating standalone claims, users increasingly prefer grounded analyses that pair claims with verifiable citations from source documents to enable independent validation. However, evaluating such analytical claims typically requires extensive expert annotation, which is costly and difficult to scale, and real-world financial analysis commonly involves long context-question-answer triplets, further increasing task complexity. To address these challenges and benchmark the current landscape of grounded analysis by LLMs, we propose a numeric evidence evaluation method that enables groundedness assessment without reliance on expert annotation. We also introduce an automated dataset construction pipeline and construct ECTs-100 from the top 100 constituents of the S&P 500 to support benchmark of both groundedness and correctness. In addition, we examine conscious incompetence, a practical failure mode in financial analysis in which LLMs must detect when available evidence is insufficient and refrain from producing unsupported hallucinations. Empirical results show that LLMs perform well in groundedness but face notable limitations in correctness, with informational insufficiency presenting an additional challenge.