mlsanityData to 14 Oct 2026

Task Weighting in Meta-learning with Trajectory Optimisation

arXiv 2301.01400 ↗First posted 4 Jan 2023cs.LG cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Developing meta-learning algorithms that are un-biased toward a subset of training tasks often requires hand-designed criteria to weight tasks, potentially resulting in sub-optimal solutions. In this paper, we introduce a new principled and fully-automated task-weighting algorithm for meta-learning methods. By considering the weights of tasks within the same mini-batch as an action, and the meta-parameter of interest as the system state, we cast the task-weighting meta-learning problem to a trajectory optimisation and employ the iterative linear quadratic regulator to determine the optimal action or weights of tasks. We theoretically show that the proposed algorithm converges to an ε0-stationary point, and empirically demonstrate that the proposed approach out-performs common hand-engineering weighting methods in two few-shot learning benchmarks.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 29 of its 65 references that are in the map. First posted on arXiv 4 Jan 2023.

Builds on

29 of 65 references.

References · 29

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  4. 2017Prototypical Networks for Few-shot Learningcited by 3,015
  5. 2016Matching Networks for One Shot Learningcited by 2,255
  6. 2018On First-Order Meta-Learning Algorithmscited by 1,018
  7. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 834
  8. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 821
  9. 2018Taskonomy: Disentangling Task Transfer Learningcited by 634
  10. 2014Convex Optimization: Algorithms and Complexitycited by 575
  11. 2017Meta-SGD: Learning to Learn Quickly for Few-Shot Learningcited by 379
  12. 2019Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examplescited by 315
17 more
  1. 2019A Baseline for Few-Shot Image Classificationcited by 298
  2. 2019Task2Vec: Task Embedding for Meta-Learningcited by 205
  3. 2020ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learningcited by 202
  4. 2018Bayesian Model-Agnostic Meta-Learningcited by 194
  5. 2019Adaptive Gradient-Based Meta-Learning Methodscited by 183
  6. 2017Practical Gauss-Newton Optimisation for Deep Learningcited by 142
  7. 2019On the Convergence Theory of Gradient-Based Model-Agnostic Meta-Learning Algorithmscited by 132
  8. 2019Infinite Mixture Prototypes for Few-Shot Learningcited by 99
  9. 2019Meta-Learning with Warped Gradient Descentcited by 98
  10. 2020Meta-Learning with Adaptive Hyperparameterscited by 79
  11. 2019BackPACK: Packing more into backpropcited by 78
  12. 2022Auto-Lambda: Disentangling Dynamic Task Relationshipscited by 73
  13. 2019Learning to Balance: Bayesian Meta-Learning for Imbalanced and Out-of-distribution Taskscited by 48
  14. 2019A Principled Approach for Learning Task Similarity in Multitask Learningcited by 32
  15. 2019Uncertainty in Model-Agnostic Meta-Learning using Variational Inferencecited by 23
  16. 2021Probabilistic task modelling for meta-learningcited by 4
  17. 2020Weighted Meta-Learningcited by 2

Built on by

Cited by year

Citing papers per year: 2024: 2.

2 papers.

  1. 2024GO4Align: Group Optimization for Multi-Task Alignmentcited by 11
  2. 2024Meta-Learning with Heterogeneous Taskscited by 2