# Improved Multi-label Classification under Temporal Concept Drift: Rethinking Group-Robust Algorithms in a Label-Wise Setting

arXiv: https://arxiv.org/abs/2203.07856 · HTML: https://mlsanity.com/paper/2203.07856/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 49 references that are in the map.
- First posted on arXiv 15 Mar 2022.

## Builds on (most cited first, 20 of 20)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [Out-of-Distribution Generalization via Risk Extrapolation (REx)](https://mlsanity.com/paper/2003.00688/) (2020)
- [Gradient Starvation: A Learning Proclivity in Neural Networks](https://mlsanity.com/paper/2011.09468/) (2020)
- [Explainable Prediction of Medical Codes from Clinical Text](https://mlsanity.com/paper/1802.05695/) (2018)
- [Well-Read Students Learn Better: On the Importance of Pre-training Compact Models](https://mlsanity.com/paper/1908.08962/) (2019)
- [AttentionXML: Label Tree-based Attention-Aware Deep Model for High-Performance Extreme Multi-Label Text Classification](https://mlsanity.com/paper/1811.01727/) (2018)
- [SSMBA: Self-Supervised Manifold Based Data Augmentation for Improving Out-of-Domain Robustness](https://mlsanity.com/paper/2009.10195/) (2020)
- [MultiEURLEX -- A multi-lingual and multi-label legal document classification dataset for zero-shot cross-lingual transfer](https://mlsanity.com/paper/2109.00904/) (2021)
- [We Need to Talk About Random Splits](https://mlsanity.com/paper/2005.00636/) (2020)
- [An Empirical Study on Large-Scale Multi-Label Text Classification Including Few and Zero-Shot Labels](https://mlsanity.com/paper/2010.01653/) (2020)
- [MultiFiT: Efficient Multi-lingual Language Model Fine-tuning](https://mlsanity.com/paper/1909.04761/) (2019)
- [Few-Shot Cross-Lingual Stance Detection with Sentiment-Based Pre-Training](https://mlsanity.com/paper/2109.06050/) (2021)
- [MESA: Boost Ensemble Imbalanced Learning with MEta-SAmpler](https://mlsanity.com/paper/2010.08830/) (2020)
- [Overview of BioASQ 2021: The ninth BioASQ challenge on Large-Scale Biomedical Semantic Indexing and Question Answering](https://mlsanity.com/paper/2106.14885/) (2021)
- [Fairness-aware Class Imbalanced Learning](https://mlsanity.com/paper/2109.10444/) (2021)
- [Measure Twice, Cut Once: Quantifying Bias and Fairness in Deep Neural Networks](https://mlsanity.com/paper/2110.04397/) (2021)

## Built on by (most cited first, 23 of 23)

- [A Multi-Task Benchmark for Korean Legal Language Understanding and Judgement Prediction](https://mlsanity.com/paper/2206.05224/) (2022)
- [An Exploration of Encoder-Decoder Approaches to Multi-Label Classification for Legal and Biomedical Text](https://mlsanity.com/paper/2305.05627/) (2023)
- [Dynamic Benchmarking of Masked Language Models on Temporal Concept Drift with Multiple Views](https://mlsanity.com/paper/2302.12297/) (2023)
- [It's about Time: Rethinking Evaluation on Rumor Detection Benchmarks using Chronological Splits](https://mlsanity.com/paper/2302.03147/) (2023)
- [On the Impact of Temporal Concept Drift on Model Explanations](https://mlsanity.com/paper/2210.09197/) (2022)
- [Rather a Nurse than a Physician -- Contrastive Explanations under Investigation](https://mlsanity.com/paper/2310.11906/) (2023)
- [ChronosLex: Time-aware Incremental Training for Temporal Generalization of Legal Classification Tasks](https://mlsanity.com/paper/2405.14211/) (2024)
- [An Effective Deployment of Contrastive Learning in Multi-label Text Classification](https://mlsanity.com/paper/2212.00552/) (2022)
- [On the Interplay between Fairness and Explainability](https://mlsanity.com/paper/2310.16607/) (2023)
- [PILOT: Legal Case Outcome Prediction with Case Law](https://mlsanity.com/paper/2401.15770/) (2024)
- [Examining Temporal Bias in Abusive Language Detection](https://mlsanity.com/paper/2309.14146/) (2023)
- [Towards Explainability in Legal Outcome Prediction Models](https://mlsanity.com/paper/2403.16852/) (2024)
- [Exploring Contrastive Learning for Long-Tailed Multi-Label Text Classification](https://mlsanity.com/paper/2404.08720/) (2024)
- [Examining Temporalities on Stance Detection towards COVID-19 Vaccination](https://mlsanity.com/paper/2304.04806/) (2023)
- [Retrieval-augmented Multi-label Text Classification](https://mlsanity.com/paper/2305.13058/) (2023)
- [Legal-Tech Open Diaries: Lesson learned on how to develop and deploy light-weight models in the era of humongous Language Models](https://mlsanity.com/paper/2210.13086/) (2022)
- [Examining the Limitations of Computational Rumor Detection Models Trained on Static Datasets](https://mlsanity.com/paper/2309.11576/) (2023)
- [A Comprehensive Review of Machine Learning Advances on Data Change: A Cross-Field Perspective](https://mlsanity.com/paper/2402.12627/) (2024)
- [Knowledge-driven Augmentation and Retrieval for Integrative Temporal Adaptation](https://mlsanity.com/paper/2604.22098/) (2026)
- [Optimized Deep Learning Models for Malware Detection under Concept Drift](https://mlsanity.com/paper/2308.10821/) (2023)
- [A Framework for Monitoring and Retraining Language Models in Real-World Applications](https://mlsanity.com/paper/2311.09930/) (2023)
- [Multi-Label Bayesian Active Learning with Inter-Label Relationships](https://mlsanity.com/paper/2411.17941/) (2024)
- [UK-PRBENCH: A Paragraph-Level Precedent Retrieval Benchmark for United Kingdom Case Law](https://mlsanity.com/paper/2609.24613/) (2026)

## Abstract (arXiv)

In document classification for, e.g., legal and biomedical text, we often deal with hundreds of classes, including very infrequent ones, as well as temporal concept drift caused by the influence of real world events, e.g., policy changes, conflicts, or pandemics. Class imbalance and drift can sometimes be mitigated by resampling the training data to simulate (or compensate for) a known target distribution, but what if the target distribution is determined by unknown future events? Instead of simply resampling uniformly to hedge our bets, we focus on the underlying optimization algorithms used to train such document classifiers and evaluate several group-robust optimization algorithms, initially proposed to mitigate group-level disparities. Reframing group-robust algorithms as adaptation algorithms under concept drift, we find that Invariant Risk Minimization and Spectral Decoupling outperform sampling-based approaches to class imbalance and concept drift, and lead to much better performance on minority classes. The effect is more pronounced the larger the label set.