mlsanityData to 14 Oct 2026

Theoretical Investigations and Practical Enhancements on Tail Task Risk Minimization in Meta Learning

arXiv 2410.22788 ↗First posted 30 Oct 2024cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Meta learning is a promising paradigm in the era of large models and task distributional robustness has become an indispensable consideration in real-world scenarios. Recent advances have examined the effectiveness of tail task risk minimization in fast adaptation robustness improvement citepwang2023simple. This work contributes to more theoretical investigations and practical enhancements in the field. Specifically, we reduce the distributionally robust strategy to a max-min optimization problem, constitute the Stackelberg equilibrium as the solution concept, and estimate the convergence rate. In the presence of tail risk, we further derive the generalization bound, establish connections with estimated quantiles, and practically improve the studied strategy. Accordingly, extensive evaluations demonstrate the significance of our proposal and its scalability to multimodal large models in boosting robustness.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 51 of its 87 references that are in the map. First posted on arXiv 30 Oct 2024.

Builds on

51 of 87 references.

References · 51

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  5. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  6. 2017Prototypical Networks for Few-shot Learningcited by 3,015
  7. 2021Convex optimizationcited by 2,515
  8. 2016Matching Networks for One Shot Learningcited by 2,255
  9. 2019Natural Adversarial Examplescited by 1,408
  10. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,256
  11. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,013
  12. 2020Meta-Learning in Neural Networks: A Surveycited by 950
38 more
  1. 2022MaPLe: Multi-modal Prompt Learningcited by 687
  2. 2021Carbon Emissions and Large Neural Network Trainingcited by 567
  3. 2019Meta-Learning with Implicit Gradientscited by 525
  4. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 503
  5. 2019Meta-Learning with Differentiable Convex Optimizationcited by 461
  6. 2018Meta-Learning for Semi-Supervised Few-Shot Classificationcited by 456
  7. 2018Conditional Neural Processescited by 426
  8. 2018Neural Processescited by 311
  9. 2019Optimizing Millions of Hyperparameters by Implicit Differentiationcited by 304
  10. 2021Scaling Laws for Transfercited by 272
  11. 2019On Gradient Descent Ascent for Nonconvex-Concave Minimax Problemscited by 270
  12. 2018Probabilistic Model-Agnostic Meta-Learningcited by 256
  13. 2018Recasting Gradient-Based Meta-Learning as Hierarchical Bayescited by 180
  14. 2023Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Modelscited by 154
  15. 2021Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyondcited by 153
  16. 2018Meta-Learning Probabilistic Inference For Predictioncited by 110
  17. 2019Infinite Mixture Prototypes for Few-Shot Learningcited by 99
  18. 2020Meta-Learning Stationary Stochastic Process Prediction with Convolutional Neural Processescited by 76
  19. 2021Medically Aware GPT-3 as a Data Generator for Medical Dialogue Summarizationcited by 76
  20. 2023LLMCarbon: Modeling the end-to-end Carbon Footprint of Large Language Modelscited by 72
  21. 2019Meta-Curvaturecited by 70
  22. 2022Sharp-MAML: Sharpness-Aware Model-Agnostic Meta Learningcited by 58
  23. 2019Learning-to-Learn Stochastic Gradient Descent with Biased Regularizationcited by 49
  24. 2021Generalization Bounds For Meta-Learning: An Information-Theoretic Analysiscited by 45
  25. 2022Hypernetworks in Meta-Reinforcement Learningcited by 42
  26. 2020Bootstrapping Neural Processescited by 40
  27. 2023Counterfactual Conservative Q Learning for Offline Multi-agent Reinforcement Learningcited by 37
  28. 2020Doubly Stochastic Variational Inference for Neural Processes with Hierarchical Latent Variablescited by 33
  29. 2020How Important is the Train-Validation Split in Meta-Learning?cited by 30
  30. 2021On Fast Adversarial Robustness Adaptation in Model-Agnostic Meta-Learningcited by 29
  31. 2023Supported Trust Region Optimization for Offline Reinforcement Learningcited by 20
  32. 2020Function Contrastive Learning of Transferable Meta-Representationscited by 19
  33. 2021Model-based Meta Reinforcement Learning using Graph Structured Surrogate Modelscited by 17
  34. 2024Robust Fast Adaptation from Adversarially Explicit Task Distribution Generationcited by 13
  35. 2024Hokoff: Real Game Dataset from Honor of Kings and its Offline Reinforcement Learning Benchmarkscited by 11
  36. 2023A Simple Yet Effective Strategy to Robustify the Meta Learning Paradigmcited by 10
  37. 2022Terminology-aware Medical Dialogue Generationcited by 8
  38. 2023Episodic Multi-Task Learning with Heterogeneous Neural Processescited by 8

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025Model Predictive Task Sampling for Efficient and Robust Adaptationcited by 21
  2. 2025Fast and Robust: Task Sampling with Posterior and Diversity Synergies for Adaptive Decision-Makers in Randomized Environmentscited by 12
  3. 2026PERO: Efficient Robust Post-Training Foundation Models for Encrypted Traffic Classificationcited by 0
  4. 2026Bridging Risk Approximation Gaps in Model Predictive Task Sampling via In-Context Modelingcited by 0