# Examining and Adapting Time for Multilingual Classification via Mixture of Temporal Experts

arXiv: https://arxiv.org/abs/2502.08825 · HTML: https://mlsanity.com/paper/2502.08825/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 55 references that are in the map.
- First posted on arXiv 12 Feb 2025.

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [Experience Replay for Continual Learning](https://mlsanity.com/paper/1811.11682/) (2018)
- [LexGLUE: A Benchmark Dataset for Legal Language Understanding in English](https://mlsanity.com/paper/2110.00976/) (2021)
- [Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approach](https://mlsanity.com/paper/1909.00161/) (2019)
- [Time-Aware Language Models as Temporal Knowledge Bases](https://mlsanity.com/paper/2106.15110/) (2021)
- [A Brief Review of Domain Adaptation](https://mlsanity.com/paper/2010.03978/) (2020)
- [ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning](https://mlsanity.com/paper/2304.05613/) (2023)
- [mGPT: Few-Shot Learners Go Multilingual](https://mlsanity.com/paper/2204.07580/) (2022)
- [Large Language Models Can Learn Temporal Reasoning](https://mlsanity.com/paper/2401.06853/) (2024)
- [Text Classification via Large Language Models](https://mlsanity.com/paper/2305.08377/) (2023)
- [MultiEURLEX -- A multi-lingual and multi-label legal document classification dataset for zero-shot cross-lingual transfer](https://mlsanity.com/paper/2109.00904/) (2021)
- [Domain Adaptation for Time Series Under Feature and Label Shifts](https://mlsanity.com/paper/2302.03133/) (2023)
- [Temporal Adaptation of BERT and Performance on Downstream Document Classification: Insights from Social Media](https://mlsanity.com/paper/2104.08116/) (2021)
- [Temporal Effects on Pre-trained Models for Language Processing Tasks](https://mlsanity.com/paper/2111.12790/) (2021)
- [ChatGPT may Pass the Bar Exam soon, but has a Long Way to Go for the LexGLUE benchmark](https://mlsanity.com/paper/2304.12202/) (2023)
- [Domain Adaptation for Time-Series Classification to Mitigate Covariate Shift](https://mlsanity.com/paper/2204.03342/) (2022)
- [TEILP: Time Prediction over Knowledge Graphs via Logical Reasoning](https://mlsanity.com/paper/2312.15816/) (2023)
- [Time-Stamped Language Model: Teaching Language Models to Understand the Flow of Events](https://mlsanity.com/paper/2104.07635/) (2021)
- [Contributions of Transformer Attention Heads in Multi- and Cross-lingual Tasks](https://mlsanity.com/paper/2108.08375/) (2021)
- [ChronosLex: Time-aware Incremental Training for Temporal Generalization of Legal Classification Tasks](https://mlsanity.com/paper/2405.14211/) (2024)
- [Prototypical Fine-tuning: Towards Robust Performance Under Varying Data Sizes](https://mlsanity.com/paper/2211.13638/) (2022)
- [Comparison between parameter-efficient techniques and full fine-tuning: A case study on multilingual news article classification](https://mlsanity.com/paper/2308.07282/) (2023)
- [Measuring, Predicting and Visualizing Short-Term Change in Word Representation and Usage in VKontakte Social Network](https://mlsanity.com/paper/1703.07012/) (2017)
- [Time Matters: Examine Temporal Effects on Biomedical Language Models](https://mlsanity.com/paper/2407.17638/) (2024)

## Built on by (most cited first, 9 of 9)

- [Safeguarding LLM Agents against Long-Horizon Threats via Shadow Memory](https://mlsanity.com/paper/2605.03228/) (2026)
- [Attributes as Textual Genes: Leveraging LLMs as Genetic Algorithm Simulators for Conditional Synthetic Data Generation](https://mlsanity.com/paper/2509.02040/) (2025)
- [Deliberate Planning in Language Models with Symbolic Representation](https://mlsanity.com/paper/2505.01479/) (2025)
- [Model-Agnostic Meta Learning for Class Imbalance Adaptation](https://mlsanity.com/paper/2604.18759/) (2026)
- [Knowledge-driven Augmentation and Retrieval for Integrative Temporal Adaptation](https://mlsanity.com/paper/2604.22098/) (2026)
- [MELON: Multimodal Mixture-of-Experts with Spectral-Temporal Fusion for Long-Term Mobility Estimation in Critical Care](https://mlsanity.com/paper/2503.11695/) (2025)
- [StepGap: A Hybrid NLI-LLM Checker for Step-Level Evidence-Gap Detectionin Multi-Hop Question Answering](https://mlsanity.com/paper/2605.24733/) (2026)
- [What Makes a Medical Checker Trainable? Diagnosing Signal Collapse and Reward Hacking in Checker-Guided RAG for Biomedical QA](https://mlsanity.com/paper/2605.25988/) (2026)
- [AIA2: Attribute-Agnostic Imbalance Augmentation for Subgroup Robustness](https://mlsanity.com/paper/2608.30297/) (2026)

## Abstract (arXiv)

Time is implicitly embedded in classification process: classifiers are usually built on existing data while to be applied on future data whose distributions (e.g., label and token) may change. However, existing state-of-the-art classification models merely consider the temporal variations and primarily focus on English corpora, which leaves temporal studies less explored, let alone under multilingual settings. In this study, we fill the gap by treating time as domains (e.g., 2024 vs. 2025), examining temporal effects, and developing a domain adaptation framework to generalize classifiers over time on multiple languages. Our framework proposes Mixture of Temporal Experts (MoTE) to leverage both semantic and data distributional shifts to learn and adapt temporal trends into classification models. Our analysis shows classification performance varies over time across different languages, and we experimentally demonstrate that MoTE can enhance classifier generalizability over temporal data shifts. Our study provides analytic insights and addresses the need for time-aware models that perform robustly in multilingual scenarios.