Learning Set Functions with Implicit Differentiation
Not on the map. Why?
Abstract (arXiv)
Ou et al. (2022) introduce the problem of learning set functions from data generated by a so-called optimal subset oracle. Their approach approximates the underlying utility function with an energy-based model, whose parameters are estimated via mean-field variational inference. Ou et al. (2022) show this reduces to fixed point iterations; however, as the number of iterations increases, automatic differentiation quickly becomes computationally prohibitive due to the size of the Jacobians that are stacked during backpropagation. We address this challenge with implicit differentiation and examine the convergence conditions for the fixed-point iterations. We empirically demonstrate the efficiency of our method on synthetic and real-world subset selection applications including product recommendation, set anomaly detection and compound selection tasks.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 35 of its 74 references that are in the map. First posted on arXiv 15 Dec 2024.
Builds on
35 of 74 references.
References · 35
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2013Intriguing properties of neural networkscited by 5,114
- 2018Neural Ordinary Differential Equationscited by 3,733
- 2014Deep Learning Face Attributes in the Wildcited by 3,679
- 2016Variational Inference: A Review for Statisticianscited by 1,466
- 2017Deep Setscited by 1,394
- 2019Deep Equilibrium Modelscited by 603
- 2018Lipschitz regularity of deep neural networks: analysis and efficient estimationcited by 413
- 2018Regularisation of Neural Networks by Enforcing Lipschitz Continuitycited by 311
- 2019Optimizing Millions of Hyperparameters by Implicit Differentiationcited by 303
- 2020Scalable Gradients for Stochastic Differential Equationscited by 264
23 more
- 2020Monotone operator equilibrium networkscited by 100
- 2019On the Limitations of Representing Functions on Setscited by 89
- 2021SetVAE: Learning Hierarchical Composition for Generative Modeling of Set-Structured Datacited by 78
- 2021Amortized Implicit Differentiation for Stochastic Bilevel Optimizationcited by 62
- 2021Stochastic Batch Acquisition: A Simple Baseline for Deep Active Learningcited by 36
- 2021Control-Oriented Model-Based Reinforcement Learning with Implicit Differentiationcited by 35
- 2020Implicit differentiation of Lasso-type models for hyperparameter optimizationcited by 34
- 2022Beyond backpropagation: bilevel optimization through implicit differentiation and equilibrium propagationcited by 26
- 2021Cross-Market Product Recommendationcited by 22
- 2014Expectation-Maximization for Learning Determinantal Point Processescited by 19
- 2021Multiset-Equivariant Set Prediction with Approximate Implicit Differentiationcited by 17
- 2022Set Norm and Equivariant Skip Connections: Putting the Deep in Deep Setscited by 17
- 2022Neural Set Function Extensions: Learning with Discrete Functions in High Dimensionscited by 14
- 2017Deep Submodular Functionscited by 11
- 2022Learning Neural Set Functions Under the Optimal Subset Oraclecited by 10
- 2020Deep Submodular Networks for Extractive Data Summarizationcited by 8
- 2023Revisiting Implicit Differentiation for Learning Problems in Optimal Controlcited by 8
- 2016Learning Influence Functions from Incomplete Observationscited by 7
- 2020Learning Set Functions that are Sparse in Non-Orthogonal Fourier Basescited by 7
- 2019Powerset Convolutional Neural Networkscited by 6
- 2021SCHA-VAE: Hierarchical Context Aggregation for Few-Shot Generationcited by 6
- 2022Neural Estimation of Submodular Functions with Applications to Differentiable Subset Selectioncited by 6
- 2013Learning Coverage Functions and Private Release of Marginalscited by 5
Built on by
Citing papers per year: 2026: 1.
1 paper.