# Building Patient Journeys in Hebrew: A Language Model for Clinical Timeline Extraction

arXiv: https://arxiv.org/abs/2512.11502 · HTML: https://mlsanity.com/paper/2512.11502/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 31 references that are in the map.
- First posted on arXiv 12 Dec 2025.

## Builds on (most cited first, 20 of 20)

- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease prediction](https://mlsanity.com/paper/2005.12833/) (2020)
- [A Multi-Axis Annotation Scheme for Event Temporal Relations](https://mlsanity.com/paper/1804.07828/) (2018)
- [Unpacking Tokenization: Evaluating Text Compression and its Correlation with Model Performance](https://mlsanity.com/paper/2403.06265/) (2024)
- [Joint Event and Temporal Relation Extraction with Shared Representations and Structured Prediction](https://mlsanity.com/paper/1909.05360/) (2019)
- [ECONET: Effective Continual Pretraining of Language Models for Event Temporal Reasoning](https://mlsanity.com/paper/2012.15283/) (2020)
- [Parameter-Efficient Fine-Tuning of LLaMA for the Clinical Domain](https://mlsanity.com/paper/2307.03042/) (2023)
- [Adapt-and-Distill: Developing Small, Fast and Effective Pretrained Language Models for Domains](https://mlsanity.com/paper/2106.13474/) (2021)
- [An Improved Neural Baseline for Temporal Relation Extraction](https://mlsanity.com/paper/1909.00429/) (2019)
- [Efficiently Adapting Pretrained Language Models To New Languages](https://mlsanity.com/paper/2311.05741/) (2023)
- [Deep Structured Neural Network for Event Temporal Relation Extraction](https://mlsanity.com/paper/1909.10094/) (2019)
- [On the Domain Adaptation and Generalization of Pretrained Language Models: A Survey](https://mlsanity.com/paper/2211.03154/) (2022)
- [Clinical Temporal Relation Extraction with Probabilistic Soft Logic Regularization and Global Inference](https://mlsanity.com/paper/2012.08790/) (2020)
- [AVocaDo: Strategy for Adapting Vocabulary to Downstream Domain](https://mlsanity.com/paper/2110.13434/) (2021)
- [Extracting Temporal Event Relation with Syntax-guided Graph Transformer](https://mlsanity.com/paper/2104.09570/) (2021)
- [Extracting or Guessing? Improving Faithfulness of Event Temporal Relation Extraction](https://mlsanity.com/paper/2210.04992/) (2022)
- [Adapting LLMs to Hebrew: Unveiling DictaLM 2.0 with Enhanced Vocabulary and Instruction Capabilities](https://mlsanity.com/paper/2407.07080/) (2024)
- [Will LLMs Replace the Encoder-Only Models in Temporal Relation Classification?](https://mlsanity.com/paper/2410.10476/) (2024)
- [Beyond Pairwise: Global Zero-shot Temporal Graph Generation](https://mlsanity.com/paper/2502.11114/) (2025)

## Abstract (arXiv)

We present a new Hebrew medical language model designed to extract structured clinical timelines from electronic health records, enabling the construction of patient journeys. Our model is based on DictaBERT 2.0 and continually pre-trained on over five million de-identified hospital records. To evaluate its effectiveness, we introduce two new datasets -- one from internal medicine and emergency departments, and another from oncology -- annotated for event temporal relations. Our results show that our model achieves strong performance on both datasets. We also find that vocabulary adaptation improves token efficiency and that de-identification does not compromise downstream performance, supporting privacy-conscious model development. The model is made available for research use under ethical restrictions.