mlsanityData to 7 Oct 2026

Robust Fast Adaptation from Adversarially Explicit Task Distribution Generation

arXiv 2407.19523 ↗First posted 28 Jul 2024cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

Meta-learning is a practical learning paradigm to transfer skills across tasks from a few examples. Nevertheless, the existence of task distribution shifts tends to weaken meta-learners' generalization capability, particularly when the training task distribution is naively hand-crafted or based on simple priors that fail to cover critical scenarios sufficiently. Here, we consider explicitly generative modeling task distributions placed over task identifiers and propose robustifying fast adaptation from adversarial training. Our approach, which can be interpreted as a model of a Stackelberg game, not only uncovers the task structure during problem-solving from an explicit generative model but also theoretically increases the adaptation robustness in worst cases. This work has practical implications, particularly in dealing with task distribution shifts in meta-learning, and contributes to theoretical insights in the field. Our method demonstrates its robustness in the presence of task subpopulation shifts and improved performance over SOTA baselines in extensive experiments. The code is available at the project site https://sites.google.com/view/ar-metalearn.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 59 of its 98 references that are in the map. First posted on arXiv 28 Jul 2024.

Builds on

59 of 98 references.

References · 59

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  4. 2013Auto-Encoding Variational Bayescited by 3,980
  5. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  6. 2015Variational Inference with Normalizing Flowscited by 2,161
  7. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  8. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,232
  9. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  10. 2021Transformers in Vision: A Surveycited by 1,009
  11. 2016HyperNetworkscited by 1,003
  12. 2020Meta-Learning in Neural Networks: A Surveycited by 944
38 more
  1. 2016Adversarial Machine Learning at Scalecited by 934
  2. 2019Meta-Learning with Implicit Gradientscited by 523
  3. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
  4. 2018Meta-Learning with Latent Embedding Optimizationcited by 476
  5. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
  6. 2020Measuring Robustness to Natural Distribution Shifts in Image Classificationcited by 452
  7. 2018Conditional Neural Processescited by 421
  8. 2018Neural Processescited by 307
  9. 2019Attentive Neural Processescited by 280
  10. 2017Meta Networkscited by 268
  11. 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 260
  12. 2021Efficiently Identifying Task Groupings for Multi-Task Learningcited by 218
  13. 2023Sequential Modeling Enables Scalable Learning for Large Vision Modelscited by 212
  14. 2020Adaptive Risk Minimization: Learning to Adapt to Domain Shiftcited by 183
  15. 2022Evolving Curricula with Regret-Based Environment Designcited by 166
  16. 2019Active Domain Randomizationcited by 161
  17. 2019Convolutional Conditional Neural Processescited by 139
  18. 2021Generating Useful Accident-Prone Driving Scenarios via a Learned Traffic Priorcited by 136
  19. 2019On the Convergence Theory of Gradient-Based Model-Agnostic Meta-Learning Algorithmscited by 132
  20. 2019Fast and Flexible Multi-Task Classification Using Conditional Neural Adaptive Processescited by 131
  21. 2019Infinite Mixture Prototypes for Few-Shot Learningcited by 99
  22. 2020Context-aware Dynamics Model for Generalization in Model-Based Reinforcement Learningcited by 95
  23. 2021SceneGen: Learning to Generate Realistic Traffic Scenescited by 83
  24. 2021Mining Latent Classes for Few-shot Segmentationcited by 69
  25. 2021Meta-Learning to Compositionally Generalizecited by 62
  26. 2020Meta-Learning Requires Meta-Augmentationcited by 57
  27. 2020Improving Generalization in Meta-learning via Task Augmentationcited by 55
  28. 2020Cautious Adaptation For Reinforcement Learning in Safety-Critical Settingscited by 42
  29. 2020Data Augmentation for Meta-Learningcited by 38
  30. 2021Understanding Continual Learning Settings with Data Distribution Drift Analysiscited by 38
  31. 2020Doubly Stochastic Variational Inference for Neural Processes with Hierarchical Latent Variablescited by 33
  32. 2021Meta Discovery: Learning to Discover Novel Classes given Very Limited Datacited by 30
  33. 2020Adaptive Task Sampling for Meta-Learningcited by 29
  34. 2021Meta-learning with an Adaptive Task Schedulercited by 27
  35. 2022Learning to Generalize across Domains on Single Test Samplescited by 24
  36. 2024Any-Shift Prompting for Generalization over Distributionscited by 24
  37. 2020On the Global Optimality of Model-Agnostic Meta-Learningcited by 21
  38. 2019What is Local Optimality in Nonconvex-Nonconcave Minimax Optimization?cited by 19

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 5; 2026: 1.

12 papers.

  1. 2024Beyond Model Adaptation at Test Time: A Surveycited by 55
  2. 2024Adaptive teachers for amortized samplerscited by 23
  3. 2024Doubly Mild Generalization for Offline Reinforcement Learningcited by 21
  4. 2024Offline Reinforcement Learning with OOD State Correction and OOD Action Suppressioncited by 17
  5. 2025Fast and Robust: Task Sampling with Posterior and Diversity Synergies for Adaptive Decision-Makers in Randomized Environmentscited by 11
  6. 2025Selective Learning for Deep Time Series Forecastingcited by 8
  7. 2025Flow-based Domain Randomization for Learning and Sequencing Robotic Skillscited by 4
  8. 2024Learning to Learn with Contrastive Meta-Objectivecited by 3
  9. 2024Theoretical Investigations and Practical Enhancements on Tail Task Risk Minimization in Meta Learningcited by 3
  10. 2025DRESS: Disentangled Representation-based Self-Supervised Meta-Learning for Diverse Taskscited by 2
  11. 2025Gains: Fine-grained Federated Domain Adaptation in Open Setcited by 0
  12. 2026PERO: Efficient Robust Post-Training Foundation Models for Encrypted Traffic Classificationcited by 0