mlsanityData to 14 Oct 2026

Large Language Models for Predictive Analysis: How Far Are They?

arXiv 2505.17149 ↗First posted 22 May 2025cs.CL cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Predictive analysis is a cornerstone of modern decision-making, with applications in various domains. Large Language Models (LLMs) have emerged as powerful tools in enabling nuanced, knowledge-intensive conversations, thus aiding in complex decision-making tasks. With the burgeoning expectation to harness LLMs for predictive analysis, there is an urgent need to systematically assess their capability in this domain. However, there is a lack of relevant evaluations in existing studies. To bridge this gap, we introduce the PredictiQ benchmark, which integrates 1130 sophisticated predictive analysis queries originating from 44 real-world datasets of 8 diverse fields. We design an evaluation protocol considering text analysis, code generation, and their alignment. Twelve renowned LLMs are evaluated, offering insights into their practical use in predictive analysis. Generally, we believe that existing LLMs still face considerable challenges in conducting predictive analysis. See hrefhttps://github.com/Cqkkkkkk/PredictiQGithub.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 31 of its 50 references that are in the map. First posted on arXiv 22 May 2025.

Builds on

31 of 50 references.

References · 31

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  5. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  6. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  7. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  8. 2023Large Language Models are not Fair Evaluatorscited by 1,085
  9. 2024Phi-4 Technical Reportcited by 711
  10. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 686
  11. 2023Understanding the Effects of RLHF on LLM Generalisation and Diversitycited by 396
  12. 2023Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluationcited by 385
19 more
  1. 2022CodeT: Code Generation with Generated Testscited by 384
  2. 2024Justice or Prejudice? Quantifying Biases in LLM-as-a-Judgecited by 368
  3. 2024Data Interpreter: An LLM Agent For Data Sciencecited by 225
  4. 2023Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verificationcited by 166
  5. 2024Large Language Models are Inconsistent and Biased Evaluatorscited by 153
  6. 2022Large Language Models are few(1)-shot Table Reasonerscited by 116
  7. 2020An Evaluation of Change Point Detection Algorithmscited by 63
  8. 2024ChatGLM-Math: Improving Math Problem-Solving in Large Language Models with a Self-Critique Pipelinecited by 47
  9. 2023Text2Analysis: A Benchmark of Table Question Answering with Advanced Data Analysis and Unclear Queriescited by 38
  10. 2023Large Language Models as Data Preprocessorscited by 34
  11. 2024Evaluating the Evaluator: Measuring LLMs' Adherence to Task Evaluation Instructionscited by 34
  12. 2023Large Language Model Prediction Capabilities: Evidence from a Real-World Forecasting Tournamentcited by 24
  13. 2023Querying Large Language Models with SQLcited by 14
  14. 2024Can Language Models Use Forecasting Strategies?cited by 13
  15. 2023Humans vs Large Language Models: Judgmental Forecasting in an Era of Advanced AIcited by 11
  16. 2025DAGPrompT: Pushing the Limits of Graph Prompting with a Distribution-aware Graph Prompt Tuning Approachcited by 11
  17. 2024HiRoPE: Length Extrapolation for Code Models Using Hierarchical Positioncited by 8
  18. 2023TreeMAN: Tree-enhanced Multimodal Attention Network for ICD Codingcited by 6
  19. 2024Integrating Generative AI into Financial Market Prediction for Improved Decision Makingcited by 4

Built on by

Cited by year

Citing papers per year: 2025: 3.

3 papers.

  1. 2025Spectral Insights into Data-Oblivious Critical Layers in Large Language Modelscited by 4
  2. 2025Representation Interventions Enable Lifelong Knowledge Memory Control in LLMscited by 3
  3. 2025SheetDesigner: MLLM-Powered Spreadsheet Layout Generation with Rule-Based and Vision-Based Reflectioncited by 1