# Language Models Can Improve Event Prediction by Few-Shot Abductive Reasoning

arXiv: https://arxiv.org/abs/2305.16646 · HTML: https://mlsanity.com/paper/2305.16646/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Placed in Causal reasoning in LLMs, one of 17 directions in LLM reasoning.
- Builds on 41 of its 86 references that are in the map.
- First posted on arXiv 26 May 2023.

## Placed in

- LLM reasoning / [Causal reasoning in LLMs](https://mlsanity.com/lr/causal-reasoning-in-llms/)

## Builds on (most cited first, 30 of 41)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Least-to-Most Prompting Enables Complex Reasoning in Large Language Models](https://mlsanity.com/paper/2205.10625/) (2022)
- [Scaling Laws for Reward Model Overoptimization](https://mlsanity.com/paper/2210.10760/) (2022)
- [Transformer Hawkes Process](https://mlsanity.com/paper/2002.09291/) (2020)
- [Know-Evolve: Deep Temporal Reasoning for Dynamic Knowledge Graphs](https://mlsanity.com/paper/1705.05742/) (2017)
- [Intensity-Free Learning of Temporal Point Processes](https://mlsanity.com/paper/1909.12127/) (2019)
- [Fully Neural Network based Model for General Temporal Point Processes](https://mlsanity.com/paper/1905.09690/) (2019)
- [Transformer Embeddings of Irregularly Spaced Events and Their Participants](https://mlsanity.com/paper/2201.00044/) (2021)
- [A Multi-Axis Annotation Scheme for Event Temporal Relations](https://mlsanity.com/paper/1804.07828/) (2018)
- [LAMBADA: Backward Chaining for Automated Reasoning in Natural Language](https://mlsanity.com/paper/2212.13894/) (2022)
- [TORQUE: A Reading Comprehension Dataset of Temporal Ordering Questions](https://mlsanity.com/paper/2005.00242/) (2020)
- [Joint Reasoning for Temporal and Causal Relations](https://mlsanity.com/paper/1906.04941/) (2019)
- [Cross-media Structured Common Space for Multimedia Event Extraction](https://mlsanity.com/paper/2005.02472/) (2020)
- [Temporal Reasoning on Implicit Events from Distant Supervision](https://mlsanity.com/paper/2010.12753/) (2020)
- [HYPRO: A Hybridly Normalized Probabilistic Model for Long-Horizon Prediction of Event Sequences](https://mlsanity.com/paper/2210.01753/) (2022)
- [Joint Constrained Learning for Event-Event Relation Extraction](https://mlsanity.com/paper/2010.06727/) (2020)
- [Identifying Coordinated Accounts on Social Media through Hidden Influence and Group Behaviours](https://mlsanity.com/paper/2008.11308/) (2020)
- [Temporal Common Sense Acquisition with Minimal Supervision](https://mlsanity.com/paper/2005.04304/) (2020)
- [Neural Temporal Point Processes For Modelling Electronic Health Records](https://mlsanity.com/paper/2007.13794/) (2020)
- [ESTER: A Machine Reading Comprehension Dataset for Event Semantic Relation Reasoning](https://mlsanity.com/paper/2104.08350/) (2021)
- [Self-Attentive Hawkes Processes](https://mlsanity.com/paper/1907.07561/) (2019)
- [Counterfactual Neural Temporal Point Process for Estimating Causal Influence of Misinformation on Social Media](https://mlsanity.com/paper/2210.07518/) (2022)
- [Generic Temporal Reasoning with Differential Analysis and Explanation](https://mlsanity.com/paper/2212.10467/) (2022)
- [User-Dependent Neural Sequence Models for Continuous-Time Event Data](https://mlsanity.com/paper/2011.03231/) (2020)
- [Learning Network of Multivariate Hawkes Processes: A Time Series Approach](https://mlsanity.com/paper/1603.04319/) (2016)

## Built on by (most cited first, 30 of 45)

- [Foundation Models for Time Series Analysis: A Tutorial and Survey](https://mlsanity.com/paper/2403.14735/) (2024)
- [Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlook](https://mlsanity.com/paper/2310.10196/) (2023)
- [EasyTPP: Towards Open Benchmarking Temporal Point Processes](https://mlsanity.com/paper/2307.08097/) (2023)
- [Empowering Time Series Analysis with Large Language Models: A Survey](https://mlsanity.com/paper/2402.03182/) (2024)
- [Large Language Models for Forecasting and Anomaly Detection: A Systematic Literature Review](https://mlsanity.com/paper/2402.10350/) (2024)
- [Enhancing Recommender Systems with Large Language Model Reasoning Graphs](https://mlsanity.com/paper/2308.10835/) (2023)
- [TimeXL: Explainable Multi-modal Time Series Prediction with LLM-in-the-Loop](https://mlsanity.com/paper/2503.01013/) (2025)
- [Prompt-augmented Temporal Point Process for Streaming Event Sequence](https://mlsanity.com/paper/2310.04993/) (2023)
- [Foundation Models for Spatio-Temporal Data Science: A Tutorial and Survey](https://mlsanity.com/paper/2503.13502/) (2025)
- [IDEA: Enhancing the Rule Learning Ability of Large Language Model Agent through Induction, Deduction, and Abduction](https://mlsanity.com/paper/2408.10455/) (2024)
- [Towards Explainable Traffic Flow Prediction with Large Language Models](https://mlsanity.com/paper/2404.02937/) (2024)
- [BRIDGE: Bootstrapping Text to Control Time-Series Generation via Multi-Agent Iterative Optimization and Diffusion Modeling](https://mlsanity.com/paper/2503.02445/) (2025)
- [Chain-of-History Reasoning for Temporal Knowledge Graph Forecasting](https://mlsanity.com/paper/2402.14382/) (2024)
- [SHIELD: LLM-Driven Schema Induction for Predictive Analytics in EV Battery Supply Chain Disruptions](https://mlsanity.com/paper/2408.05357/) (2024)
- [A Survey on Mathematical Reasoning and Optimization with Large Language Models](https://mlsanity.com/paper/2503.17726/) (2025)
- [Unraveling Spatio-Temporal Foundation Models via the Pipeline Lens: A Comprehensive Review](https://mlsanity.com/paper/2506.01364/) (2025)
- [Can Prompts Rewind Time for LLMs? Evaluating the Effectiveness of Prompted Knowledge Cutoffs](https://mlsanity.com/paper/2510.02340/) (2025)
- [Assessing the Reasoning Capabilities of LLMs in the context of Evidence-based Claim Verification](https://mlsanity.com/paper/2402.10735/) (2024)
- [Demonstration of DB-GPT: Next Generation Data Interaction System Empowered by Large Language Models](https://mlsanity.com/paper/2404.10209/) (2024)
- [Latent Logic Tree Extraction for Event Sequence Explanation from LLMs](https://mlsanity.com/paper/2406.01124/) (2024)
- [TPP-LLM: Modeling Temporal Point Processes by Efficiently Fine-Tuning Large Language Models](https://mlsanity.com/paper/2410.02062/) (2024)
- [A Comprehensive Evaluation of Large Language Models on Temporal Event Forecasting](https://mlsanity.com/paper/2407.11638/) (2024)
- [When Reasoning Meets Information Aggregation: A Case Study with Sports Narratives](https://mlsanity.com/paper/2406.12084/) (2024)
- [DanmakuTPPBench: A Multi-modal Benchmark for Temporal Point Process Modeling and Understanding](https://mlsanity.com/paper/2505.18411/) (2025)
- [Byte-token Enhanced Language Models for Temporal Point Processes Analysis](https://mlsanity.com/paper/2502.07139/) (2025)
- [Back to the Future: Towards Explainable Temporal Reasoning with Large Language Models](https://mlsanity.com/paper/2310.01074/) (2023)
- [Wiring the 'Why': A Unified Taxonomy and Survey of Abductive Reasoning in LLMs](https://mlsanity.com/paper/2604.08016/) (2026)
- [ONSEP: A Novel Online Neural-Symbolic Framework for Event Prediction Based on Large Language Model](https://mlsanity.com/paper/2408.07840/) (2024)
- [Foundation Model Sherpas: Guiding Foundation Models through Knowledge and Reasoning](https://mlsanity.com/paper/2402.01602/) (2024)
- [Advances in Temporal Point Processes: Bayesian, Neural, and LLM Approaches](https://mlsanity.com/paper/2501.14291/) (2025)

## Abstract (arXiv)

Large language models have shown astonishing performance on a wide range of reasoning tasks. In this paper, we investigate whether they could reason about real-world events and help improve the prediction performance of event sequence models. We design LAMP, a framework that integrates a large language model in event prediction. Particularly, the language model performs abductive reasoning to assist an event sequence model: the event model proposes predictions on future events given the past; instructed by a few expert-annotated demonstrations, the language model learns to suggest possible causes for each proposal; a search module finds out the previous events that match the causes; a scoring function learns to examine whether the retrieved events could actually cause the proposal. Through extensive experiments on several challenging real-world datasets, we demonstrate that our framework -- thanks to the reasoning capabilities of large language models -- could significantly outperform the state-of-the-art event sequence models.