mlsanityData to 7 Oct 2026

Time Matters: Examine Temporal Effects on Biomedical Language Models

arXiv 2407.17638 ↗First posted 24 Jul 2024cs.CLcited by 7

Not on the map. Why?

Abstract (arXiv)

Time roots in applying language models for biomedical applications: models are trained on historical data and will be deployed for new or future data, which may vary from training data. While increasing biomedical tasks have employed state-of-the-art language models, there are very few studies have examined temporal effects on biomedical models when data usually shifts across development and deployment. This study fills the gap by statistically probing relations between language model performance and data shifts across three biomedical tasks. We deploy diverse metrics to evaluate model performance, distance methods to measure data drifts, and statistical methods to quantify temporal effects on biomedical language models. Our study shows that time matters for deploying biomedical language models, while the degree of performance degradation varies by biomedical tasks and statistical quantification approaches. We believe this study can establish a solid benchmark to evaluate and assess temporal effects on deploying biomedical language models.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 17 of its 43 references that are in the map. First posted on arXiv 24 Jul 2024.

Builds on

17 of 43 references.

References · 17

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2019Publicly Available Clinical BERT Embeddingscited by 995
  5. 2018Universal Sentence Encodercited by 485
  6. 2019Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasetscited by 298
  7. 2023MedCPT: Contrastive Pre-trained Transformers with Large-scale PubMed Search Logs for Zero-shot Biomedical Information Retrievalcited by 182
  8. 2023Do We Still Need Clinical Language Models?cited by 97
  9. 2021Pre-trained Language Models in Biomedical Domain: A Systematic Surveycited by 87
  10. 2024DALK: Dynamic Co-Augmentation of LLMs and KG to answer Alzheimer's Disease Questions with Scientific Literaturecited by 50
  11. 2022A Multimodal Transformer: Fusing Clinical Notes with Structured EHR Data for Interpretable In-Hospital Mortality Predictioncited by 48
  12. 2021Temporal Effects on Pre-trained Models for Language Processing Taskscited by 46
5 more
  1. 2020BioALBERT: A Simple and Effective Pre-trained Language Model for Biomedical Named Entity Recognitioncited by 19
  2. 2024Thought Graph: Generating Thought Process for Biological Reasoningcited by 10
  3. 2023Token Imbalance Adaptation for Radiology Report Generationcited by 9
  4. 2024Length-Aware Multi-Kernel Transformer for Long Document Classificationcited by 5
  5. 2023Evaluating Model Performance in Medical Datasets Over Timecited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 3; 2026: 2.

7 papers.

  1. 2025Protein Large Language Models: A Comprehensive Surveycited by 18
  2. 2025Examining and Adapting Time for Multilingual Classification via Mixture of Temporal Expertscited by 9
  3. 2024Predicting 30-Day Hospital Readmission in Medicare Patients: Insights from an LSTM Deep Learning Modelcited by 8
  4. 2026Safeguarding LLM Agents against Long-Horizon Threats via Shadow Memorycited by 6
  5. 2024Examining Imbalance Effects on Performance and Demographic Fairness of Clinical Language Modelscited by 4
  6. 2025Affective-ROPTester: Capability and Bias Analysis of LLMs in Predicting Retinopathy of Prematuritycited by 3
  7. 2026Knowledge-driven Augmentation and Retrieval for Integrative Temporal Adaptationcited by 1