# Rockafellian Relaxation and Stochastic Optimization under Perturbations

arXiv: https://arxiv.org/abs/2204.04762 · HTML: https://mlsanity.com/paper/2204.04762/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 74 references that are in the map.
- First posted on arXiv 10 Apr 2022.

## Builds on (most cited first, 18 of 18)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [Data-driven Distributionally Robust Optimization Using the Wasserstein Metric: Performance Guarantees and Tractable Reformulations](https://mlsanity.com/paper/1505.05116/) (2015)
- [Statistics of Robust Optimization: A Generalized Empirical Likelihood Approach](https://mlsanity.com/paper/1610.03425/) (2016)
- [Data-Driven Robust Optimization](https://mlsanity.com/paper/1401.0212/) (2013)
- [Let's Make Block Coordinate Descent Converge Faster: Faster Greedy Rules, Message-Passing, Active-Set Complexity, and Superlinear Convergence](https://mlsanity.com/paper/1712.08859/) (2017)
- [Diametrical Risk Minimization: Theory and Computations](https://mlsanity.com/paper/1910.10844/) (2019)
- [Optimistic Distributionally Robust Optimization for Nonparametric Likelihood Approximation](https://mlsanity.com/paper/1910.10583/) (2019)
- [Optimistic Distributionally Robust Policy Optimization](https://mlsanity.com/paper/2006.07815/) (2020)
- [Optimistic Robust Optimization With Applications To Machine Learning](https://mlsanity.com/paper/1711.07511/) (2017)
- [On sample average approximation for two-stage stochastic programs without relatively complete recourse](https://mlsanity.com/paper/1912.13078/) (2019)
- [A data-driven approach to beating SAA out-of-sample](https://mlsanity.com/paper/2105.12342/) (2021)
- [Fabrication-Adaptive Optimization, with an Application to Photonic Crystal Design](https://mlsanity.com/paper/1307.5571/) (2013)
- [Good and Bad Optimization Models: Insights from Rockafellians](https://mlsanity.com/paper/2105.06073/) (2021)
- [Trimmed Constrained Mixed Effects Models: Formulations and Algorithms](https://mlsanity.com/paper/1909.10700/) (2019)

## Built on by (most cited first, 2 of 2)

- [On Tractability, Complexity, and Mixed-Integer Convex Programming Representability of Distributionally Favorable Optimization](https://mlsanity.com/paper/2401.17899/) (2024)
- [Mitigating the Impact of Labeling Errors on Training via Rockafellian Relaxation](https://mlsanity.com/paper/2405.20531/) (2024)

## Abstract (arXiv)

In practice, optimization models are often prone to unavoidable inaccuracies due to dubious assumptions and corrupted data. Traditionally, this placed special emphasis on risk-based and robust formulations, and their focus on ``conservative" decisions. We develop, in contrast, an ``optimistic" framework based on Rockafellian relaxations in which optimization is conducted not only over the original decision space but also jointly with a choice of model perturbation. The framework enables us to address challenging problems with ambiguous probability distributions from the areas of two-stage stochastic optimization without relatively complete recourse, probability functions lacking continuity properties, expectation constraints, and outlier analysis. We are also able to circumvent the fundamental difficulty in stochastic optimization that convergence of distributions fails to guarantee convergence of expectations. The framework centers on the novel concepts of exact and limit-exact Rockafellians, with interpretations of ``negative'' regularization emerging in certain settings. We illustrate the role of Phi-divergence, examine rates of convergence under changing distributions, and explore extensions to first-order optimality conditions. The main development is free of assumptions about convexity, smoothness, and even continuity of objective functions. Numerical results in the setting of computer vision and text analytics with label noise illustrate the framework.