A Comprehensive Review of Machine Learning Advances on Data Change: A Cross-Field Perspective
Not on the map. Why?
Abstract (arXiv)
Recent artificial intelligence (AI) technologies show remarkable evolution in various academic fields and industries. However, in the real world, dynamic data lead to principal challenges for deploying AI models. An unexpected data change brings about severe performance degradation in AI models. We identify two major related research fields, domain shift and concept drift according to the setting of the data change. Although these two popular research fields aim to solve distribution shift and non-stationary data stream problems, the underlying properties remain similar which also encourages similar technical approaches. In this review, we regroup domain shift and concept drift into a single research problem, namely the data change problem, with a systematic overview of state-of-the-art methods in the two research fields. We propose a three-phase problem categorization scheme to link the key ideas in the two technical fields. We thus provide a novel scope for researchers to explore contemporary technical strategies, learn industrial applications, and identify future directions for addressing data change challenges.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 124 of its 212 references that are in the map. First posted on arXiv 20 Feb 2024.
Builds on
124 of 212 references.
References · 124
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
- 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
- 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
- 2015Domain-Adversarial Training of Neural Networkscited by 3,023
- 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
- 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
- 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
- 2019Invariant Risk Minimizationcited by 1,832
- 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,366
38 more
- 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
- 2017Deep Hashing Network for Unsupervised Domain Adaptationcited by 1,181
- 2017Deeper, Broader and Artier Domain Generalizationcited by 983
- 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 863
- 2021Generalized Out-of-Distribution Detection: A Surveycited by 796
- 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 762
- 2021Domain Generalization: A Surveycited by 760
- 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 718
- 2022Out-of-Distribution Detection with Deep Nearest Neighborscited by 596
- 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 589
- 2020Learning from Noisy Labels with Deep Neural Networks: A Surveycited by 580
- 2022Continual Test-Time Domain Adaptationcited by 520
- 2021ReAct: Out-of-distribution Detection With Rectified Activationscited by 486
- 2022InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutionscited by 479
- 2020Learning under Concept Drift: A Reviewcited by 470
- 2022Efficient Test-Time Model Adaptation without Forgettingcited by 467
- 2017VisDA: The Visual Domain Adaptation Challengecited by 432
- 2018A Survey on Deep Transfer Learningcited by 417
- 2019Contrastive Adaptation Network for Unsupervised Domain Adaptationcited by 407
- 2015Return of Frustratingly Easy Domain Adaptationcited by 389
- 2019An Evaluation Dataset for Intent Classification and Out-of-Scope Predictioncited by 378
- 2018Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfacescited by 376
- 2018Detecting and Correcting for Label Shift with Black Box Predictorscited by 364
- 2023A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPTcited by 364
- 2018Learning Adversarially Fair and Transferable Representationscited by 353
- 2020Learning to Learn Single Domain Generalizationcited by 295
- 2020A Brief Review of Domain Adaptationcited by 287
- 2016Revisiting Batch Normalization For Practical Domain Adaptationcited by 285
- 2021Recent Advances in Adversarial Training for Adversarial Robustnesscited by 276
- 2017WebVision Database: Visual Learning and Understanding from Web Datacited by 262
- 2020Transfer Learning in Deep Reinforcement Learning: A Surveycited by 259
- 2022Contrastive Test-Time Adaptationcited by 258
- 2021Understanding self-supervised Learning Dynamics without Contrastive Pairscited by 254
- 2015Domain Generalization for Object Recognition with Multi-task Autoencoderscited by 253
- 2021Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotationscited by 253
- 2019Adversarial Domain Adaptation with Domain Mixupcited by 250
- 2022Test-Time Training with Masked Autoencoderscited by 208
- 2023On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspectivecited by 208
50 most cited of 124 All 124 (JSON)
Built on by
Citing papers per year: 2024: 1.
1 paper.