# Learning Samples Importance: Parameterizing Dual Variables in Everywhere Learning

arXiv: https://arxiv.org/abs/2609.36310 · HTML: https://mlsanity.com/paper/2609.36310/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 35 references that are in the map.
- First posted on arXiv 28 Sep 2026.

## Builds on (most cited first, 22 of 22)

- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [TOFU: A Task of Fictitious Unlearning for LLMs](https://mlsanity.com/paper/2401.06121/) (2024)
- [Size-Independent Sample Complexity of Neural Networks](https://mlsanity.com/paper/1712.06541/) (2017)
- [Optimization with Non-Differentiable Constraints with Applications to Fairness, Recall, Churn, and Other Goals](https://mlsanity.com/paper/1809.04198/) (2018)
- [Predicting AC Optimal Power Flows: Combining Deep Learning and Lagrangian Dual Methods](https://mlsanity.com/paper/1909.10461/) (2019)
- [OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metrics](https://mlsanity.com/paper/2506.12618/) (2025)
- [Optimal Power Flow Using Graph Neural Networks](https://mlsanity.com/paper/1910.09658/) (2019)
- [Self-Supervised Primal-Dual Learning for Constrained Optimization](https://mlsanity.com/paper/2208.09046/) (2022)
- [Probably Approximately Correct Constrained Learning](https://mlsanity.com/paper/2006.05487/) (2020)
- [DeepOPF: A Feasibility-Optimized Deep Neural Network Approach for AC Optimal Power Flow Problems](https://mlsanity.com/paper/2007.01002/) (2020)
- [PowerModels.jl: An Open-Source Framework for Exploring Power Flow Formulations](https://mlsanity.com/paper/1711.01728/) (2017)
- [Machine Learning for AC Optimal Power Flow](https://mlsanity.com/paper/1910.08842/) (2019)
- [DeepOPF-V: Solving AC-OPF Problems Efficiently](https://mlsanity.com/paper/2103.11793/) (2021)
- [GNNs as Predictors of Agentic Workflow Performances](https://mlsanity.com/paper/2503.11301/) (2025)
- [Strong NP-hardness of AC power flows feasibility](https://mlsanity.com/paper/1512.07315/) (2015)
- [Unsupervised Optimal Power Flow Using Graph Neural Networks](https://mlsanity.com/paper/2210.09277/) (2022)
- [Convex Relaxation of Optimal Power Flow, Part II: Exactness](https://mlsanity.com/paper/1405.0814/) (2014)
- [Strong SOCP Relaxations for the Optimal Power Flow Problem](https://mlsanity.com/paper/1504.06770/) (2015)
- [Convex Relaxations of Optimal Power Flow Problems: An Illustrative Example](https://mlsanity.com/paper/1510.04330/) (2015)
- [Unsupervised Deep Learning for AC Optimal Power Flow via Lagrangian Duality](https://mlsanity.com/paper/2212.03977/) (2022)
- [Everywhere Learning: Artificial Intelligence with Pointwise Constraints](https://mlsanity.com/paper/2606.01557/) (2026)
- [Every Sample Counts: Supervised Fine-Tuning of Language Models with Pointwise Constraints](https://mlsanity.com/paper/2607.08968/) (2026)

## Abstract (arXiv)

Everywhere learning provides a principled framework for training AI models under constraints that must hold throughout the data distribution. In the dual domain, these pointwise constraints give rise to functional dual variables. In this work, we propose to learn these dual variables, motivated by the fact that their values encode useful information about the underlying constrained problem. By representing the dual variable as a parametric function of each sample, we enable the learned multiplier to be evaluated on new, unseen samples. This contrasts with standard empirical dual formulations, which assign an independent multiplier to each training sample. We characterize the error in the recovered primal solution induced by restricting the dual variable to a parametric function class and show that it is controlled by how well this class approximates the optimal statistical multiplier. Moreover, we show that the learned parametric multiplier retains the sensitivity interpretation of the optimal statistical multiplier, yielding approximate sensitivity guarantees that extend beyond the samples used for training. We empirically validate our theory across a variety of everywhere learning tasks, showing that the resulting constrained problems can be solved efficiently and that the learned dual variables provide meaningful representations of sample-level sensitivity.