# Transformer-Based Temporal Information Extraction and Application: A Review

arXiv: https://arxiv.org/abs/2504.07470 · HTML: https://mlsanity.com/paper/2504.07470/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 114 references that are in the map.
- First posted on arXiv 10 Apr 2025.

## Builds on (most cited first, 30 of 41)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Large Language Models Can Learn Temporal Reasoning](https://mlsanity.com/paper/2401.06853/) (2024)
- [A Multi-Axis Annotation Scheme for Event Temporal Relations](https://mlsanity.com/paper/1804.07828/) (2018)
- [Joint Event and Temporal Relation Extraction with Shared Representations and Structured Prediction](https://mlsanity.com/paper/1909.05360/) (2019)
- [Joint Constrained Learning for Event-Event Relation Extraction](https://mlsanity.com/paper/2010.06727/) (2020)
- [Zero-shot Temporal Relation Extraction with ChatGPT](https://mlsanity.com/paper/2304.05454/) (2023)
- [Generative Knowledge Graph Construction: A Review](https://mlsanity.com/paper/2210.12714/) (2022)
- [ECONET: Effective Continual Pretraining of Language Models for Event Temporal Reasoning](https://mlsanity.com/paper/2012.15283/) (2020)
- [An Improved Neural Baseline for Temporal Relation Extraction](https://mlsanity.com/paper/1909.00429/) (2019)
- [Deep Structured Neural Network for Event Temporal Relation Extraction](https://mlsanity.com/paper/1909.10094/) (2019)
- [Clinical Temporal Relation Extraction with Probabilistic Soft Logic Regularization and Global Inference](https://mlsanity.com/paper/2012.08790/) (2020)
- [Extracting Temporal Event Relation with Syntax-guided Graph Transformer](https://mlsanity.com/paper/2104.09570/) (2021)
- [Domain Knowledge Empowered Structured Neural Net for End-to-End Event Temporal Relation Extraction](https://mlsanity.com/paper/2009.07373/) (2020)
- [Extracting or Guessing? Improving Faithfulness of Event Temporal Relation Extraction](https://mlsanity.com/paper/2210.04992/) (2022)
- [More than Classification: A Unified Framework for Event Temporal Relation Extraction](https://mlsanity.com/paper/2305.17607/) (2023)
- [Salient Span Masking for Temporal Understanding](https://mlsanity.com/paper/2303.12860/) (2023)
- [Extracting Event Temporal Relations via Hyperbolic Geometry](https://mlsanity.com/paper/2109.05527/) (2021)
- [Severing the Edge Between Before and After: Neural Architectures for Temporal Ordering of Events](https://mlsanity.com/paper/2004.04295/) (2020)
- [Once Upon a textitTime in textitGraph: Relative-Time Pretraining for Complex Temporal Reasoning](https://mlsanity.com/paper/2310.14709/) (2023)
- [A Survey on Temporal Reasoning for Temporal Information Extraction from Text (Extended Abstract)](https://mlsanity.com/paper/2005.06527/) (2020)
- [Event Temporal Relation Extraction with Bayesian Translational Model](https://mlsanity.com/paper/2302.04985/) (2023)
- [Effective Distant Supervision for Temporal Relation Extraction](https://mlsanity.com/paper/2010.12755/) (2020)
- [Exploring Contextualized Neural Language Models for Temporal Dependency Parsing](https://mlsanity.com/paper/2004.14577/) (2020)
- [Multilingual Normalization of Temporal Expressions with Masked Language Models](https://mlsanity.com/paper/2205.10399/) (2022)

## Built on by (most cited first, 1 of 1)

- [CRAFT: LLM-Based Iterative Refinement for Temporal Reasoning over Clinical Narratives](https://mlsanity.com/paper/2608.12779/) (2026)

## Abstract (arXiv)

Temporal information extraction (IE) aims to extract structured temporal information from unstructured text, thereby uncovering the implicit timelines within. This technique is applied across domains such as healthcare, newswire, and intelligence analysis, aiding models in these areas to perform temporal reasoning and enabling human users to grasp the temporal structure of text. Transformer-based pre-trained language models have produced revolutionary advancements in natural language processing, demonstrating exceptional performance across a multitude of tasks. Despite the achievements garnered by Transformer-based approaches in temporal IE, there is a lack of comprehensive reviews on these endeavors. In this paper, we aim to bridge this gap by systematically summarizing and analyzing the body of work on temporal IE using Transformers while highlighting potential future research directions.