mlsanityData to 7 Oct 2026

HyperTime: Hyperparameter Optimization for Combating Temporal Distribution Shifts

arXiv 2305.18421 ↗First posted 28 May 2023cs.LG cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

In this work, we propose a hyperparameter optimization method named HyperTime to find hyperparameters robust to potential temporal distribution shifts in the unseen test data. Our work is motivated by an important observation that it is, in many cases, possible to achieve temporally robust predictive performance via hyperparameter optimization. Based on this observation, we leverage the `worst-case-oriented' philosophy from the robust optimization literature to help find such robust hyperparameter configurations. HyperTime imposes a lexicographic priority order on average validation loss and worst-case validation loss over chronological validation sets. We perform a theoretical analysis on the upper bound of the expected test loss, which reveals the unique advantages of our approach. We also demonstrate the strong empirical performance of the proposed method on multiple machine learning tasks with temporal distribution shifts.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 37 of its 75 references that are in the map. First posted on arXiv 28 May 2023.

Builds on

37 of 75 references.

References · 37

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  3. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  4. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  5. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  6. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  7. 2019Invariant Risk Minimizationcited by 1,832
  8. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  9. 2015Learning Transferable Features with Deep Adaptation Networkscited by 1,323
  10. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,232
  11. 2016Deep CORAL: Correlation Alignment for Deep Domain Adaptationcited by 1,194
  12. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,194
25 more
  1. 2017Continual Learning with Deep Generative Replaycited by 1,152
  2. 2018Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigencecited by 734
  3. 2016Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimizationcited by 688
  4. 2014Deep Domain Confusion: Maximizing for Domain Invariancecited by 597
  5. 2018Progress & Compress: A scalable framework for continual learningcited by 516
  6. 2018Learning Models with Uniform Performance via Distributionally Robust Optimizationcited by 334
  7. 2017Certifying Some Distributional Robustness with Principled Adversarial Trainingcited by 333
  8. 2019Adversarial Domain Adaptation with Domain Mixupcited by 250
  9. 2019Domain Randomization and Pyramid Consistency: Simulation-to-Real Generalization without Accessing Target Domain Datacited by 245
  10. 2022Improving Out-of-Distribution Robustness via Selective Augmentationcited by 202
  11. 2023How to Index Item IDs for Recommendation Foundation Modelscited by 102
  12. 2022Wild-Time: A Benchmark of in-the-Wild Distribution Shift over Timecited by 85
  13. 2013Data-Driven Robust Optimizationcited by 84
  14. 2019A Comparative Analysis of XGBoostcited by 80
  15. 2022Connect, Not Collapse: Explaining Contrastive Learning for Unsupervised Domain Adaptationcited by 66
  16. 2015A Century of Portraits: A Visual Historical Record of American High School Yearbookscited by 54
  17. 2021The Power of Contrast for Feature Learning: A Theoretical Analysiscited by 53
  18. 2021Reproducibility Report: La-MAML: Look-ahead Meta Learning for Continual Learningcited by 42
  19. 2019Continual Learning with Adaptive Weights (CLAW)cited by 38
  20. 2023UP5: Unbiased Foundation Model for Fairness-aware Recommendationcited by 38
  21. 2021NAS-OoD: Neural Architecture Search for Out-of-Distribution Generalizationcited by 28
  22. 2022Shifts 2.0: Extending The Dataset of Real Distributional Shiftscited by 22
  23. 2021You Only Compress Once: Towards Effective and Elastic BERT Compression via Exploit-Explore Stochastic Nature Gradientcited by 14
  24. 2019DDPNAS: Efficient Neural Architecture Search via Dynamic Distribution Pruningcited by 13
  25. 2020SiamParseNet: Joint Body Parsing and Label Propagation in Infant Movement Videoscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 3; 2025: 0; 2026: 1.

8 papers.

  1. 2023MathChat: Converse to Tackle Challenging Math Problems with LLM Agentscited by 67
  2. 2024StateFlow: Enhancing LLM Task-Solving through State-Driven Workflowscited by 65
  3. 2024Cantor: Inspiring Multimodal Chain-of-Thought of MLLMcited by 34
  4. 2023IDEAL: Influence-Driven Selective Annotations Empower In-Context Learners in Large Language Modelscited by 26
  5. 2024EcoAct: Economic Agent Determines When to Register What Actioncited by 11
  6. 2023How Many Validation Labels Do You Need? Exploring the Design Space of Label-Efficient Model Rankingcited by 8
  7. 2023General Phrase Debiaser: Debiasing Masked Language Models at a Multi-Token Levelcited by 2
  8. 2026A nonparametric two-sample test using a parametric integral probability metriccited by 0