Generalization Error of f-Divergence Stabilized Algorithms via Duality

The solution to empirical risk minimization with f-divergence regularization (ERM-fDR) is extended to constrained optimization problems, establishing conditions for equivalence between the solution and constraints. A dual formulation of ERM-fDR is introduced, providing a computationally efficient method to derive the normalization function of the ERM-fDR solution. This dual approach leverages the Legendre-Fenchel transform and the implicit function theorem, enabling explicit characterizations of the generalization error for general algorithms under mild conditions, and another for ERM-fDR solutions.