mlsanityData to 7 Oct 2026

Learning-to-learn non-convex piecewise-Lipschitz functions

arXiv 2108.08770 ↗First posted 19 Aug 2021cs.LGcited by 15

Not on the map. Why?

Abstract (arXiv)

We analyze the meta-learning of the initialization and step-size of learning algorithms for piecewise-Lipschitz functions, a non-convex setting with applications to both machine learning and algorithms. Starting from recent regret bounds for the exponential forecaster on losses with dispersed discontinuities, we generalize them to be initialization-dependent and then use this result to propose a practical meta-learning procedure that learns both the initialization and the step-size of the algorithm from multiple online learning tasks. Asymptotically, we guarantee that the average regret across tasks scales with a natural notion of task-similarity that measures the amount of overlap between near-optimal regions of different tasks. Finally, we instantiate the method and its guarantee in two important settings: robust meta-learning and multi-task data-driven algorithm design.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 34 of its 60 references that are in the map. First posted on arXiv 19 Aug 2021.

Builds on

34 of 60 references.

References · 34

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  2. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
  3. 2018How To Backdoor Federated Learningcited by 745
  4. 2019Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAMLcited by 359
  5. 2019Robust Aggregation for Federated Learningcited by 314
  6. 2020Data Poisoning Attacks Against Federated Learning Systemscited by 266
  7. 2017One-Shot Imitation Learningcited by 232
  8. 2020Few-Shot Learning via Learning the Representation, Provablycited by 213
  9. 2015The Benefit of Multitask Representation Learningcited by 194
  10. 2019Adaptive Gradient-Based Meta-Learning Methodscited by 183
  11. 2020Provable Meta-Learning of Linear Representationscited by 161
  12. 2019On the Convergence Theory of Gradient-Based Model-Agnostic Meta-Learning Algorithmscited by 132
22 more
  1. 2016Coin Betting and Parameter-Free Online Learningcited by 131
  2. 2019Online Control with Adversarial Disturbancescited by 92
  3. 2018Learning to Branchcited by 78
  4. 2019Provable Guarantees for Gradient-Based Meta-Learningcited by 67
  5. 2021Federated Hyperparameter Tuning: Challenges, Baselines, and Connections to Weight-Sharingcited by 58
  6. 2015A PAC Approach to Application-Specific Algorithm Selectioncited by 57
  7. 2019Learning-to-Learn Stochastic Gradient Descent with Biased Regularizationcited by 49
  8. 2020Robust Meta-learning for Mixed Linear Regression with Small Batchescited by 41
  9. 2019Differentially Private Meta-Learningcited by 40
  10. 2017Dispersion for Data-Driven Algorithm Design, Online Learning, and Private Optimizationcited by 37
  11. 2016Learning-Theoretic Foundations of Algorithm Configuration for Combinatorial Partitioning Problemscited by 30
  12. 2014Efficient Representations for Life-Long Learning and Autoencodingcited by 23
  13. 2016Regret Bounds for Lifelong Learningcited by 19
  14. 2018Data-Driven Clustering via Parameterized Lloyd's Familiescited by 17
  15. 2020A Sample Complexity Separation between Non-Convex and Convex Meta-Learningcited by 17
  16. 2021Meta-Learning with Neural Tangent Kernelscited by 15
  17. 2019Semi-bandit Optimization in the Dispersed Settingcited by 14
  18. 2019Learning to Linkcited by 13
  19. 2020Guarantees for Tuning the Step Size using a Learning-to-Learn Approachcited by 10
  20. 2018Adversarial Vision Challengecited by 9
  21. 2018Adversarial Online Learning with noisecited by 7
  22. 2016Online Optimization of Smoothed Piecewise Constant Functionscited by 6

Built on by

Cited by year

Citing papers per year: 2022: 4; 2023: 2; 2024: 4; 2025: 5.

15 papers.

Sort:

  1. 2022Learning Predictions for Algorithms with Predictionscited by 25
  2. 2022Provably tuning the ElasticNet across instancescited by 14
  3. 2024A CMDP-within-online framework for Meta-Safe Reinforcement Learningcited by 12
  4. 2024Algorithm Configuration for Structured Pfaffian Settingscited by 10
  5. 2023Meta-Learning Adversarial Bandit Algorithmscited by 8
  6. 2022Meta-Learning Adversarial Banditscited by 5
  7. 2022Learning-Augmented Private Algorithms for Multiple Quantile Releasecited by 5
  8. 2023New Guarantees for Learning Revenue Maximizing Menus of Lotteries and Two-Part Tariffscited by 4
  9. 2024Competitive strategies to use "warm start" algorithms with predictionscited by 4
  10. 2024Learning accurate and interpretable tree-based modelscited by 3
  11. 2025How Many Domains Suffice for Domain Generalization? A Tight Characterization via the Domain Shattering Dimensioncited by 3
  12. 2025Learning in Structured Stackelberg Gamescited by 2
3 more
  1. 2025Conservative classifiers do consistently well with improving agents: characterizing statistical and online learningcited by 2
  2. 2025Distribution-dependent Generalization Bounds for Tuning Linear Regression Across Taskscited by 2
  3. 2025Gradient Descent with Provably Tuned Learning-rate Schedulescited by 1