# EconLogicQA: A Question-Answering Benchmark for Evaluating Large Language Models in Economic Sequential Reasoning

arXiv: https://arxiv.org/abs/2405.07938 · HTML: https://mlsanity.com/paper/2405.07938/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 14 of its 14 references that are in the map.
- First posted on arXiv 13 May 2024.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)

## Builds on (most cited first, 14 of 14)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge](https://mlsanity.com/paper/1811.00937/) (2018)
- [Yi: Open Foundation Models by 01.AI](https://mlsanity.com/paper/2403.04652/) (2024)
- [PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Change](https://mlsanity.com/paper/2206.10498/) (2022)
- [Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoning](https://mlsanity.com/paper/1909.00277/) (2019)
- [WHEN FLUE MEETS FLANG: Benchmarks and Large Pre-trained Language Model for Financial Domain](https://mlsanity.com/paper/2211.00083/) (2022)
- [CLadder: Assessing Causal Reasoning in Language Models](https://mlsanity.com/paper/2312.04350/) (2023)
- [BBT-Fin: Comprehensive Construction of Chinese Financial Domain Pre-trained Language Model, Corpus and Benchmark](https://mlsanity.com/paper/2302.09432/) (2023)
- [AQA-Bench: An Interactive Benchmark for Evaluating LLMs' Sequential Reasoning Ability](https://mlsanity.com/paper/2402.09404/) (2024)
- [STEPS: A Benchmark for Order Reasoning in Sequential Tasks](https://mlsanity.com/paper/2306.04441/) (2023)

## Built on by (most cited first, 22 of 22)

- [A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challenges](https://mlsanity.com/paper/2406.11903/) (2024)
- [Fino1: On the Transferability of Reasoning-Enhanced LLMs and Reinforcement Learning to Finance](https://mlsanity.com/paper/2502.08127/) (2025)
- [InvAgent: A Large Language Model based Multi-Agent System for Inventory Management in Supply Chains](https://mlsanity.com/paper/2407.11384/) (2024)
- [FAMMA: A Benchmark for Financial Domain Multilingual Multimodal Question Answering](https://mlsanity.com/paper/2410.04526/) (2024)
- [Standard Benchmarks Fail -- Auditing LLM Agents in Finance Must Prioritize Risk](https://mlsanity.com/paper/2502.15865/) (2025)
- [Inferring Events from Time Series using Language Models](https://mlsanity.com/paper/2503.14190/) (2025)
- [Distributive Fairness in Large Language Models: Evaluating Alignment with Human Values](https://mlsanity.com/paper/2502.00313/) (2025)
- [EconWebArena: Benchmarking Autonomous Agents on Economic Tasks in Realistic Web Environments](https://mlsanity.com/paper/2506.08136/) (2025)
- [TREX: Automating LLM Fine-tuning via Agent-Driven Tree-based Exploration](https://mlsanity.com/paper/2604.14116/) (2026)
- [J&H: Evaluating the Robustness of Large Language Models Under Knowledge-Injection Attacks in Legal Domain](https://mlsanity.com/paper/2503.18360/) (2025)
- [CRMAgent: A Multi-Agent LLM System for E-Commerce CRM Message Template Generation](https://mlsanity.com/paper/2507.08325/) (2025)
- [Leveraging Large Language Models for Risk Assessment in Hyperconnected Logistic Hub Network Deployment](https://mlsanity.com/paper/2503.21115/) (2025)
- [Reasoning Like an Economist: Post-Training on Economic Problems Induces Strategic Generalization in LLMs](https://mlsanity.com/paper/2506.00577/) (2025)
- [EconSkills: Studying Skill Transfer and Retrieval for Web Agents on Live Economic Data](https://mlsanity.com/paper/2609.19523/) (2026)
- [Improving LLMs with a knowledge from databases](https://mlsanity.com/paper/2506.05560/) (2025)
- [Evaluating LLMs' Reasoning Over Ordered Procedural Steps](https://mlsanity.com/paper/2511.04688/) (2025)
- [AfriEconQA: A Benchmark for Quantitative and Temporal Reasoning over World Bank Economic Reports](https://mlsanity.com/paper/2601.15297/) (2026)
- [AI Agent Systems for Supply Chains: Structured Decision Prompts and Memory Retrieval](https://mlsanity.com/paper/2602.05524/) (2026)
- [SupChain-Bench: Benchmarking Large Language Models for Real-World Supply Chain Management](https://mlsanity.com/paper/2602.07342/) (2026)
- [Strategy Accumulation and Guided Execution for Automated LLM Fine-Tuning](https://mlsanity.com/paper/2609.22257/) (2026)
- [RSI-Master: Structuring Experiments to Guide Autonomous Model Improvement](https://mlsanity.com/paper/2609.35561/) (2026)
- [AutoMARS: Automatic and Scalable Multi-Agent Large Language Model Framework for Supply Chain Risk Assessment](https://mlsanity.com/paper/2610.17206/) (2026)

## Abstract (arXiv)

In this paper, we introduce EconLogicQA, a rigorous benchmark designed to assess the sequential reasoning capabilities of large language models (LLMs) within the intricate realms of economics, business, and supply chain management. Diverging from traditional benchmarks that predict subsequent events individually, EconLogicQA poses a more challenging task: it requires models to discern and sequence multiple interconnected events, capturing the complexity of economic logics. EconLogicQA comprises an array of multi-event scenarios derived from economic articles, which necessitate an insightful understanding of both temporal and logical event relationships. Through comprehensive evaluations, we exhibit that EconLogicQA effectively gauges a LLM's proficiency in navigating the sequential complexities inherent in economic contexts. We provide a detailed description of EconLogicQA dataset and shows the outcomes from evaluating the benchmark across various leading-edge LLMs, thereby offering a thorough perspective on their sequential reasoning potential in economic contexts. Our benchmark dataset is available at https://huggingface.co/datasets/yinzhu-quan/econ_logic_qa.