# A Dual Optimization View to Empirical Risk Minimization with f-Divergence Regularization

arXiv: https://arxiv.org/abs/2508.03314 · HTML: https://mlsanity.com/paper/2508.03314/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 47 references that are in the map.
- First posted on arXiv 5 Aug 2025.

## Builds on (most cited first, 16 of 16)

- [Tightening Mutual Information Based Bounds on Generalization Error](https://mlsanity.com/paper/1901.04609/) (2019)
- [User-friendly guarantees for the Langevin Monte Carlo with inaccurate gradient](https://mlsanity.com/paper/1710.00095/) (2017)
- [How much does your data exploration overfit? Controlling bias via information usage](https://mlsanity.com/paper/1511.05219/) (2015)
- [Generalization Error Bounds Via R'enyi-, f-Divergences and Maximal Leakage](https://mlsanity.com/paper/1912.01439/) (2019)
- [When Optimizing f-divergence is Robust with Label Noise](https://mlsanity.com/paper/2011.03687/) (2020)
- [Non-exponentially weighted aggregation: regret bounds for unbounded loss functions](https://mlsanity.com/paper/2009.03017/) (2020)
- [Empirical Risk Minimization with Relative Entropy Regularization](https://mlsanity.com/paper/2211.06617/) (2022)
- [On the Validation of Gibbs Algorithms: Training Datasets, Test Datasets and their Aggregation](https://mlsanity.com/paper/2306.12380/) (2023)
- [Empirical Risk Minimization with Relative Entropy Regularization: Optimality and Sensitivity Analysis](https://mlsanity.com/paper/2202.04385/) (2022)
- [Stochastic Gradient Langevin Dynamics with Variance Reduction](https://mlsanity.com/paper/2102.06759/) (2021)
- [Smoothed f-Divergence Distributionally Robust Optimization](https://mlsanity.com/paper/2306.14041/) (2023)
- [Equivalence of the Empirical Risk Minimization to Regularization on the Family of f-Divergences](https://mlsanity.com/paper/2402.00501/) (2024)
- [Asymmetry of the Relative Entropy in the Regularization of Empirical Risk Minimization](https://mlsanity.com/paper/2410.02833/) (2024)
- [Information-theoretic Analysis of Test Data Sensitivity in Uncertainty](https://mlsanity.com/paper/2307.12456/) (2023)
- [Regularization via f-Divergence: An Application to Multi-Oxide Spectroscopic Analysis](https://mlsanity.com/paper/2502.03755/) (2025)
- [An Information-Theoretic Analysis for Federated Learning under Concept Drift](https://mlsanity.com/paper/2506.21036/) (2025)

## Built on by (most cited first, 1 of 1)

- [Empirical Risk Minimization with f-Divergence Regularization](https://mlsanity.com/paper/2601.13191/) (2026)

## Abstract (arXiv)

The dual formulation of empirical risk minimization with f-divergence regularization (ERM-fDR) is introduced. The solution of the dual optimization problem to the ERM-fDR is connected to the notion of normalization function introduced as an implicit function. This dual approach leverages the Legendre-Fenchel transform and the implicit function theorem to provide a nonlinear ODE expression to the normalization function. Furthermore, the nonlinear ODE expression and its properties provide a computationally efficient method to calculate the normalization function of the ERM-fDR solution under a mild condition.