mlsanityData to 7 Oct 2026

Limited Marginal Benefit of Reasoning-Heavy LLM Deployment in ESG Narrative Scoring: A 4-Model Consensus Study on Japanese Listed Firms

arXiv 2606.13693 ↗First posted 22 May 2026cs.CY cs.AIcited by 0

On the map: LLM reasoning → Reasoning for agents, planning and code

Abstract (arXiv)

Automated scoring of ESG narrative disclosures with large language models (LLMs) is gaining traction, yet whether reasoning-heavy frontier models add value commensurate with their cost remains empirically unsettled. We evaluate this question on a corpus of ten Japanese listed firms across three rubric axes -- quantitative targets, progress-tracking infrastructure, and external-standard alignment -- using a four-model consensus design that combines a reasoning-on frontier model with three reasoning-off contemporaries. Across 120 firm x axis x model scores, the pooled mean absolute deviation between the reasoning-on model and each reasoning-off counterpart is 0.38 on a 5-point scale; only 2% of pairwise comparisons reach a two-point deviation, and none exceeds two points. Per-firm cost accounting shows the reasoning-on arm alone costs roughly 5.6x as much as the three-provider reasoning-off ensemble, for outcomes that differ only within small margins. We conclude that in span-based ESG narrative scoring, reasoning-heavy deployment does not materially improve outcomes relative to reasoning-off consensus, while substantially increasing operational cost. We discuss implications for cost-effective ESG auto-scoring pipelines and LLM deployment governance in applied accountability settings. An earlier version of this work is available on SSRN (Abstract ID 6683303).

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 3 of its 7 references that are in the map. First posted on arXiv 22 May 2026.

Builds on

3 of 7 references.

References · 3

  1. 2023ESGReveal: An LLM-based approach for extracting structured data from ESG reportscited by 15
  2. 2025EulerESG: Automating ESG Disclosure Analysis with LLMscited by 2
  3. 2026ESGLens: An LLM-Based RAG Framework for Interactive ESG Report Analysis and Score Predictioncited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025FinEval-KR: A Financial Domain Evaluation Framework for Large Language Models' Knowledge and Reasoning
  2. 2026Same Input, Different Scores: A Multi Model Study on the Inconsistency of LLM Judge
  3. 2026The Price of Thought: Does Test-Time Reasoning Pay in LLM Trading?
  4. 2025Accounting Reasoning in Large Language Models: Concepts, Evaluation, and Empirical Analysis
  5. 2026InvestPhilBench: A Multi-Layer Benchmark for Evaluating Large Language Model Procedural Reasoning in Expert Investment Philosophy
  6. 2026Are the Financial Reasoning from LLMs Credible? A Real World Test over Long-Horizon Statements
  7. 2026FinReasoning: A Hierarchical Benchmark for Reliable Financial Research Reporting
  8. 2026DSA: Evidence-Aware LLM-Agent Orchestration for Multi-Market Stock Research