mlsanityData to 14 Oct 2026

Simultaneous Perturbation Method for Multi-Task Weight Optimization in One-Shot Meta-Learning

arXiv 2110.13188 ↗First posted 25 Oct 2021cs.LG cs.AI cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

Meta-learning methods aim to build learning algorithms capable of quickly adapting to new tasks in low-data regime. One of the most difficult benchmarks of such algorithms is a one-shot learning problem. In this setting many algorithms face uncertainties associated with limited amount of training samples, which may result in overfitting. This problem can be resolved by providing additional information to the model. One of the most efficient ways to do this is multi-task learning. In this paper we investigate the modification of a standard meta-learning pipeline. The proposed method simultaneously utilizes information from several meta-training tasks in a common loss function. The impact of these tasks in the loss function is controlled by a per task weight. Proper optimization of the weights can have big influence on training and the final quality of the model. We propose and investigate the use of methods from the family of Simultaneous Perturbation Stochastic Approximation (SPSA) for optimization of meta-train tasks weights. We also demonstrate superiority of stochastic approximation in comparison to gradient-based method. The proposed Multi-Task Modification can be applied to almost all meta-learning methods. We study applications of this modification on Model-Agnostic Meta-Learning and Prototypical Network algorithms on CIFAR-FS, FC100, miniImageNet and tieredImageNet one-shot learning benchmarks. During these experiments Multi-Task Modification has demonstrated improvement over original methods. SPSA-Tracking algorithm first adapted in this paper for multi-task weight optimization shows the largest accuracy boost that is competitive to the state-of-the-art meta-learning methods. Our code is available online.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 28 of its 43 references that are in the map. First posted on arXiv 25 Oct 2021.

Builds on

28 of 43 references.

References · 28

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  4. 2017Prototypical Networks for Few-shot Learningcited by 3,015
  5. 2016Matching Networks for One Shot Learningcited by 2,255
  6. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,358
  7. 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,143
  8. 2020A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challengescited by 819
  9. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 803
  10. 2019A Closer Look at Few-shot Classificationcited by 684
  11. 2018Meta-Learning with Latent Embedding Optimizationcited by 476
  12. 2019Meta-Learning with Differentiable Convex Optimizationcited by 461
16 more
  1. 2018Meta-Learning for Semi-Supervised Few-Shot Classificationcited by 456
  2. 2020Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?cited by 437
  3. 2018TADAM: Task dependent adaptive metric for improved few-shot learningcited by 408
  4. 2018Meta-learning with differentiable closed-form solverscited by 397
  5. 2018Dynamic Few-Shot Visual Learning without Forgettingcited by 385
  6. 2020A Metric Learning Reality Checkcited by 321
  7. 2018Meta-Transfer Learning for Few-Shot Learningcited by 306
  8. 2019A Baseline for Few-Shot Image Classificationcited by 298
  9. 2018Probabilistic Model-Agnostic Meta-Learningcited by 256
  10. 2019Few-Shot Learning with Embedded Class Models and Shot-Free Meta Trainingcited by 57
  11. 2021Bridging Multi-Task Learning and Meta-Learning: Towards Efficient Training and Effective Adaptationcited by 50
  12. 2019Uncertainty in Model-Agnostic Meta-Learning using Variational Inferencecited by 23
  13. 2021Reinforced Attention for Few-Shot Learning and Beyondcited by 18
  14. 2017Discriminative k-shot learning using probabilistic modelscited by 12
  15. 2019Large Scale Landmark Recognition via Deep Metric Learningcited by 6
  16. 2020Simultaneous Perturbation Stochastic Approximation for Few-Shot Learningcited by 3

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025GLUE: Gradient-free Learning to Unify Expertscited by 0