mlsanityData to 7 Oct 2026

EconLogicQA: A Question-Answering Benchmark for Evaluating Large Language Models in Economic Sequential Reasoning

arXiv 2405.07938 ↗First posted 13 May 2024cs.CLcited by 21

On the map: LLM reasoning → Reasoning for agents, planning and code

Abstract (arXiv)

In this paper, we introduce EconLogicQA, a rigorous benchmark designed to assess the sequential reasoning capabilities of large language models (LLMs) within the intricate realms of economics, business, and supply chain management. Diverging from traditional benchmarks that predict subsequent events individually, EconLogicQA poses a more challenging task: it requires models to discern and sequence multiple interconnected events, capturing the complexity of economic logics. EconLogicQA comprises an array of multi-event scenarios derived from economic articles, which necessitate an insightful understanding of both temporal and logical event relationships. Through comprehensive evaluations, we exhibit that EconLogicQA effectively gauges a LLM's proficiency in navigating the sequential complexities inherent in economic contexts. We provide a detailed description of EconLogicQA dataset and shows the outcomes from evaluating the benchmark across various leading-edge LLMs, thereby offering a thorough perspective on their sequential reasoning potential in economic contexts. Our benchmark dataset is available at https://huggingface.co/datasets/yinzhu-quan/econ_logic_qa.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 14 of its 14 references that are in the map. First posted on arXiv 13 May 2024.

Builds on

14 of 14 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Change21 Jun

Other references · 13

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  6. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  7. 2024Yi: Open Foundation Models by 01.AIcited by 706
  8. 2019Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoningcited by 281
  9. 2022WHEN FLUE MEETS FLANG: Benchmarks and Large Pre-trained Language Model for Financial Domaincited by 117
  10. 2023CLadder: Assessing Causal Reasoning in Language Modelscited by 102
  11. 2023BBT-Fin: Comprehensive Construction of Chinese Financial Domain Pre-trained Language Model, Corpus and Benchmarkcited by 60
  12. 2024AQA-Bench: An Interactive Benchmark for Evaluating LLMs' Sequential Reasoning Abilitycited by 8
1 more
  1. 2023STEPS: A Benchmark for Order Reasoning in Sequential Taskscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 11; 2026: 7.

21 papers.

Sort:

By area:

  1. 2024A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challengescited by 86
  2. 2025Fino1: On the Transferability of Reasoning-Enhanced LLMs and Reinforcement Learning to Financecited by 32
  3. 2024InvAgent: A Large Language Model based Multi-Agent System for Inventory Management in Supply Chainscited by 20
  4. 2024FAMMA: A Benchmark for Financial Domain Multilingual Multimodal Question Answeringcited by 19
  5. 2025Standard Benchmarks Fail -- Auditing LLM Agents in Finance Must Prioritize Riskcited by 13
  6. 2025Inferring Events from Time Series using Language Modelscited by 10
  7. 2025Distributive Fairness in Large Language Models: Evaluating Alignment with Human Valuescited by 9
  8. 2025EconWebArena: Benchmarking Autonomous Agents on Economic Tasks in Realistic Web Environmentscited by 8
  9. 2025J&H: Evaluating the Robustness of Large Language Models Under Knowledge-Injection Attacks in Legal Domaincited by 7
  10. 2026TREX: Automating LLM Fine-tuning via Agent-Driven Tree-based Explorationcited by 6
  11. 2025CRMAgent: A Multi-Agent LLM System for E-Commerce CRM Message Template Generationcited by 5
  12. 2025Leveraging Large Language Models for Risk Assessment in Hyperconnected Logistic Hub Network Deploymentcited by 2
9 more
  1. 2025Reasoning Like an Economist: Post-Training on Economic Problems Induces Strategic Generalization in LLMscited by 2
  2. 2025Improving LLMs with a knowledge from databasescited by 0
  3. 2025Evaluating LLMs' Reasoning Over Ordered Procedural Stepscited by 0
  4. 2026AfriEconQA: A Benchmark for Quantitative and Temporal Reasoning over World Bank Economic Reportscited by 0
  5. 2026AI Agent Systems for Supply Chains: Structured Decision Prompts and Memory Retrievalcited by 0
  6. 2026SupChain-Bench: Benchmarking Large Language Models for Real-World Supply Chain Managementcited by 0
  7. 2026EconSkills: Studying Skill Transfer and Retrieval for Web Agents on Live Economic Datacited by 0
  8. 2026Strategy Accumulation and Guided Execution for Automated LLM Fine-Tuningcited by 0
  9. 2026RSI-Master: Structuring Experiments to Guide Autonomous Model Improvementcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025STEER-ME: Assessing the Microeconomic Reasoning of Large Language Models
  2. 2024EconNLI: Evaluating Large Language Models on Economics Reasoning
  3. 2023BizBench: A Quantitative Reasoning Benchmark for Business and Finance
  4. 2025Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Models
  5. 2024FCMR: Robust Evaluation of Financial Cross-Modal Multi-Hop Reasoning
  6. 2026The Reasoning Tax: Token Economics of LLM Reasoning Across Task Types and Deployment Contexts
  7. 2026QuantEval: A Benchmark for Financial Quantitative Tasks in Large Language Models
  8. 2025TransientTables: Evaluating LLMs' Reasoning on Temporally Evolving Semi-structured Tables