Data to 13 Oct 2026

Meta-learning with differentiable closed-form solvers

arXiv 1805.08136 ↗First posted 21 May 2018cs.CV cs.LG stat.MLcited by 396

Not on the map. Why?

Abstract (arXiv)

Adapting deep networks to new concepts from a few examples is challenging, due to the high computational requirements of standard fine-tuning procedures. Most work on few-shot learning has thus focused on simple learning techniques for adaptation, such as nearest neighbours or gradient descent. Nonetheless, the machine learning literature contains a wealth of methods that learn non-deep models very efficiently. In this paper, we propose to use these fast convergent methods as the main adaptation mechanism for few-shot learning. The main idea is to teach a deep network to use standard machine learning tools, such as ridge regression, as part of its own internal model, enabling it to quickly adapt to novel data. This requires back-propagating errors through the solver steps. While normally the cost of the matrix operations involved in such a process would be significant, by using the Woodbury identity we can make the small number of examples work to our advantage. We propose both closed-form and iterative solvers, based on ridge regression and logistic regression components. Our methods constitute a simple and novel approach to the problem of few-shot learning and achieve performance competitive with or superior to the state of the art on three benchmarks.

Key facts. Cited by 396 papers in the map (data through 13 Oct 2026). First posted on arXiv 21 May 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 108; 2022: 104; 2023: 77; 2024: 61; 2025: 23; 2026: 23.

396 papers.

Sort:

By area:

2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,363
2022Transformers learn in-context by gradient descentcited by 720
2022CrossPoint: Self-Supervised Cross-Modal Contrastive Learning for 3D Point Cloud Understandingcited by 198
2021Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyondcited by 151
2022Pushing the Limits of Simple Pipelines for Few-Shot Learning: External Data and Fine-Tuning Make a Differencecited by 125
2021Cycle Self-Training for Domain Adaptationcited by 120
2021Learning Accurate Dense Correspondences and When to Trust Themcited by 113
2023LoCoOp: Few-Shot Out-of-Distribution Detection via Prompt Learningcited by 107
2021Provably Faster Algorithms for Bilevel Optimizationcited by 106
2021On the Role of Neural Collapse in Transfer Learningcited by 100
2022Self-Support Few-Shot Semantic Segmentationcited by 96
2023FeCAM: Exploiting the Heterogeneity of Class Distributions in Exemplar-Free Continual Learningcited by 86

38 more

2022Tutorial on amortized optimizationcited by 83
2021Few-Shot Graph Learning for Molecular Property Predictioncited by 81
2023Generating images of rare concepts using pre-trained diffusion modelscited by 79
2021Relational Embedding for Few-Shot Classificationcited by 74
2023Advances and Challenges in Meta-Learning: A Technical Reviewcited by 65
2021Amortized Implicit Differentiation for Stochastic Bilevel Optimizationcited by 63
2021Joint Inductive and Transductive Learning for Video Object Segmentationcited by 61
2022FedNest: Federated Bilevel, Minimax, and Compositional Optimizationcited by 60
2022Sharp-MAML: Sharpness-Aware Model-Agnostic Meta Learningcited by 57
2021Exploring Complementary Strengths of Invariant and Equivariant Representations for Few-Shot Learningcited by 56
2021Few-shot Conformal Prediction with Auxiliary Taskscited by 53
2023PiMAE: Point Cloud and Image Interactive Masked Autoencoders for 3D Object Detectioncited by 53
2021Pareto Self-Supervised Training for Few-Shot Learningcited by 52
2021Bridging Multi-Task Learning and Meta-Learning: Towards Efficient Training and Effective Adaptationcited by 50
2021CombOptNet: Fit the Right NP-Hard Problem by Learning Integer Programming Constraintscited by 49
2021Meta-Learning with Task-Adaptive Loss Function for Few-Shot Learningcited by 49
2022Generalizing to New Physical Systems via Context-Informed Dynamics Modelcited by 49
2021Shallow Bayesian Meta Learning for Real-World Few-Shot Recognitioncited by 48
2021Lower Bounds and Accelerated Algorithms for Bilevel Optimizationcited by 46
2022Synergies between Disentanglement and Sparsity: Generalization and Identifiability in Multi-Task Learningcited by 44
2023Norm-guided latent space exploration for text-to-image generationcited by 43
2021How Fine-Tuning Allows for Effective Meta-Learningcited by 41
2023First-order penalty methods for bilevel optimizationcited by 40
2021Adaptive-Control-Oriented Meta-Learning for Nonlinear Systemscited by 38
2021Partner-Assisted Learning for Few-Shot Image Classificationcited by 36
2022DDG-DA: Data Distribution Generation for Predictable Concept Drift Adaptationcited by 36
2022Will Bilevel Optimizers Benefit from Loopscited by 36
2022Rethinking Generalization in Few-Shot Classificationcited by 36
2022Learning Domain Invariant Prompt for Vision-Language Modelscited by 35
2021On the Convergence Theory for Hessian-Free Bilevel Algorithmscited by 33
2022Decentralized Gossip-Based Stochastic Bilevel Optimization over Communication Networkscited by 32
2023Leveraging sparse and shared feature activations for disentangled representation learningcited by 32
2021A Metalearning Approach for Physics-Informed Neural Networks (PINNs): Application to Parameterized PDEscited by 31
2022Few-Shot Classification with Contrastive Learningcited by 31
2023Optimal Algorithms for Stochastic Bilevel Optimization under Relaxed Smoothness Conditionscited by 31
2021On the Power of Multitask Representation Learning in Linear MDPcited by 30
2022A Closer Look at Learned Optimization: Stability, Robustness, and Inductive Biasescited by 30
2023Meta-Album: Multi-domain Meta-Dataset for Few-Shot Image Classificationcited by 30

50 most cited of 396 All 396 (JSON)