# Improving Large Language Models in Event Relation Logical Prediction

arXiv: https://arxiv.org/abs/2310.09158 · HTML: https://mlsanity.com/paper/2310.09158/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 55 references that are in the map.
- First posted on arXiv 13 Oct 2023.

## Builds on (most cited first, 26 of 26)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Large Language Models Can Self-Improve](https://mlsanity.com/paper/2210.11610/) (2022)
- [Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning](https://mlsanity.com/paper/2305.12295/) (2023)
- [ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Language](https://mlsanity.com/paper/2012.13048/) (2020)
- [Faithful Chain-of-Thought Reasoning](https://mlsanity.com/paper/2301.13379/) (2023)
- [Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4](https://mlsanity.com/paper/2304.03439/) (2023)
- [ROSCOE: A Suite of Metrics for Scoring Step-by-Step Reasoning](https://mlsanity.com/paper/2212.07919/) (2022)
- [Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanations](https://mlsanity.com/paper/2205.11822/) (2022)
- [Large Language Model Is Not a Good Few-shot Information Extractor, but a Good Reranker for Hard Samples!](https://mlsanity.com/paper/2303.08559/) (2023)
- [Exploring the Feasibility of ChatGPT for Event Extraction](https://mlsanity.com/paper/2303.03836/) (2023)
- [MAVEN-ERE: A Unified Large-scale Dataset for Event Coreference, Temporal, Causal, and Subevent Relation Extraction](https://mlsanity.com/paper/2211.07342/) (2022)
- [Joint Reasoning for Temporal and Causal Relations](https://mlsanity.com/paper/1906.04941/) (2019)
- [Joint Constrained Learning for Event-Event Relation Extraction](https://mlsanity.com/paper/2010.06727/) (2020)
- [TransOMCS: From Linguistic Graphs to Commonsense Knowledge](https://mlsanity.com/paper/2005.00206/) (2020)
- [Deep Structured Neural Network for Event Temporal Relation Extraction](https://mlsanity.com/paper/1909.10094/) (2019)
- [Clinical Temporal Relation Extraction with Probabilistic Soft Logic Regularization and Global Inference](https://mlsanity.com/paper/2012.08790/) (2020)
- [ERGO: Event Relational Graph Transformer for Document-level Event Causality Identification](https://mlsanity.com/paper/2204.07434/) (2022)
- [End-to-End Neural Event Coreference Resolution](https://mlsanity.com/paper/2009.08153/) (2020)
- [Learning Constraints and Descriptive Segmentation for Subevent Detection](https://mlsanity.com/paper/2109.06316/) (2021)

## Built on by (most cited first, 17 of 17)

- [Quantifying and Mitigating Unimodal Biases in Multimodal Large Language Models: A Causal Perspective](https://mlsanity.com/paper/2403.18346/) (2024)
- [Benchmarking Reasoning Robustness in Large Language Models](https://mlsanity.com/paper/2503.04550/) (2025)
- [Causal Evaluation of Language Models](https://mlsanity.com/paper/2405.00622/) (2024)
- [FollowEval: A Multi-Dimensional Benchmark for Assessing the Instruction-Following Capability of Large Language Models](https://mlsanity.com/paper/2311.09829/) (2023)
- [Reason from Fallacy: Enhancing Large Language Models' Logical Reasoning through Logical Fallacy Understanding](https://mlsanity.com/paper/2404.04293/) (2024)
- [Document-level Causal Relation Extraction with Knowledge-guided Binary Question Answering](https://mlsanity.com/paper/2410.04752/) (2024)
- [CELLO: Causal Evaluation of Large Vision-Language Models](https://mlsanity.com/paper/2406.19131/) (2024)
- [Synergizing Logical Reasoning, Knowledge Management and Collaboration in Multi-Agent LLM System](https://mlsanity.com/paper/2507.02170/) (2025)
- [Conditional and Modal Reasoning in Large Language Models](https://mlsanity.com/paper/2401.17169/) (2024)
- [Counterfactual-Consistency Prompting for Relative Temporal Understanding in Large Language Models](https://mlsanity.com/paper/2502.11425/) (2025)
- [Event Extraction in Large Language Model](https://mlsanity.com/paper/2512.19537/) (2025)
- [MathDivide: Improved mathematical reasoning by large language models](https://mlsanity.com/paper/2405.13004/) (2024)
- [CRAT: A Multi-Agent Framework for Causality-Enhanced Reflective and Retrieval-Augmented Translation with Large Language Models](https://mlsanity.com/paper/2410.21067/) (2024)
- [MAQInstruct: Instruction-based Unified Event Relation Extraction](https://mlsanity.com/paper/2502.03954/) (2025)
- [A Survey on Training-free Alignment of Large Language Models](https://mlsanity.com/paper/2508.09016/) (2025)
- [Synergizing LLMs and Knowledge Graphs: A Novel Approach to Software Repository-Related Question Answering](https://mlsanity.com/paper/2412.03815/) (2024)
- [From Single- to Cross-Document: Benchmarking Multi-Granularity Event Analysis of Large Language Models](https://mlsanity.com/paper/2607.27654/) (2026)

## Abstract (arXiv)

Event relations are crucial for narrative understanding and reasoning. Governed by nuanced logic, event relation extraction (ERE) is a challenging task that demands thorough semantic understanding and rigorous logical reasoning. In this paper, we conduct an in-depth investigation to systematically explore the capability of LLMs in understanding and applying event relation logic. More in detail, we first investigate the deficiencies of LLMs in logical reasoning across different tasks. Our study reveals that LLMs are not logically consistent reasoners, which results in their suboptimal performance on tasks that need rigorous reasoning. To address this, we explore three different approaches to endow LLMs with event relation logic, and thus enable them to generate more coherent answers across various scenarios. Based on our approach, we also contribute a synthesized dataset (LLM-ERL) involving high-order reasoning for evaluation and fine-tuning. Extensive quantitative and qualitative analyses on different tasks also validate the effectiveness of our approaches and provide insights for solving practical tasks with LLMs in future work. Codes are available at https://github.com/chenmeiqii/Teach-LLM-LR.