# Leaving the Nest: Going Beyond Local Loss Functions for Predict-Then-Optimize

arXiv: https://arxiv.org/abs/2305.16830 · HTML: https://mlsanity.com/paper/2305.16830/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 29 references that are in the map.
- First posted on arXiv 26 May 2023.

## Builds on (most cited first, 13 of 13)

- [Cyclical Learning Rates for Training Neural Networks](https://mlsanity.com/paper/1506.01186/) (2015)
- [Differentiable Convex Optimization Layers](https://mlsanity.com/paper/1910.12430/) (2019)
- [Smart "Predict, then Optimize"](https://mlsanity.com/paper/1710.08005/) (2017)
- [Melding the Data-Decisions Pipeline: Decision-Focused Learning for Combinatorial Optimization](https://mlsanity.com/paper/1809.05504/) (2018)
- [Differentiable MPC for End-to-end Planning and Control](https://mlsanity.com/paper/1810.13400/) (2018)
- [MIPaaL: Mixed Integer Program as a Layer](https://mlsanity.com/paper/1907.05912/) (2019)
- [Differentiable Dynamic Programming for Structured Prediction and Attention](https://mlsanity.com/paper/1802.03676/) (2018)
- [Contrastive Losses and Solution Caching for Predict-and-Optimize](https://mlsanity.com/paper/2011.05354/) (2020)
- [End to end learning and optimization on graphs](https://mlsanity.com/paper/1905.13732/) (2019)
- [Automatically Learning Compact Quality-aware Surrogates for Optimization Problems](https://mlsanity.com/paper/2006.10815/) (2020)
- [Differentiable Submodular Maximization](https://mlsanity.com/paper/1803.01785/) (2018)
- [Decision-Aware Learning for Optimizing Health Supply Chains](https://mlsanity.com/paper/2211.08507/) (2022)
- [A Note on Task-Aware Loss via Reweighing Prediction Loss by Decision-Regret](https://mlsanity.com/paper/2211.05116/) (2022)

## Built on by (most cited first, 18 of 18)

- [Gen-DFL: Decision-Focused Generative Learning for Robust Decision Making](https://mlsanity.com/paper/2502.05468/) (2025)
- [TaskMet: Task-Driven Metric Learning for Model Learning](https://mlsanity.com/paper/2312.05250/) (2023)
- [Solver-Free Decision-Focused Learning for Linear Optimization Problems](https://mlsanity.com/paper/2505.22224/) (2025)
- [Benchmarking PtO and PnO Methods in the Predictive Combinatorial Optimization Regime](https://mlsanity.com/paper/2311.07633/) (2023)
- [CaVE: A Cone-Aligned Approach for Fast Predict-then-optimize with Binary Linear Programs](https://mlsanity.com/paper/2312.07718/) (2023)
- [Return Prediction for Mean-Variance Portfolio Selection: How Decision-Focused Learning Shapes Forecasting Models](https://mlsanity.com/paper/2409.09684/) (2024)
- [Decision-Focused Fine-Tuning of Time Series Foundation Models for Dispatchable Feeder Optimization](https://mlsanity.com/paper/2503.01936/) (2025)
- [DF2: Distribution-Free Decision-Focused Learning](https://mlsanity.com/paper/2308.05889/) (2023)
- [GenCO: Generating Diverse Designs with Combinatorial Constraints](https://mlsanity.com/paper/2310.02442/) (2023)
- [Locally Convex Global Loss Network for Decision-Focused Learning](https://mlsanity.com/paper/2403.01875/) (2024)
- [Learning Joint Models of Prediction and Optimization](https://mlsanity.com/paper/2409.04898/) (2024)
- [What is the Right Notion of Distance between Predict-then-Optimize Tasks?](https://mlsanity.com/paper/2409.06997/) (2024)
- [Prediction Loss Guided Decision-Focused Learning](https://mlsanity.com/paper/2509.08359/) (2025)
- [DFF: Decision-Focused Fine-tuning for Smarter Predict-then-Optimize with Limited Data](https://mlsanity.com/paper/2501.01874/) (2025)
- [Decision-Focused Learning: When and Why Traditional Prediction Models Fail](https://mlsanity.com/paper/2606.21773/) (2026)
- [Decision-focused predictions via pessimistic bilevel optimization: complexity and algorithms](https://mlsanity.com/paper/2312.17640/) (2023)
- [Scalable Decision Focused Learning via Online Trainable Surrogates](https://mlsanity.com/paper/2512.03861/) (2025)
- [Scalable Decision-Focused Learning through Cost-Sensitive Regression](https://mlsanity.com/paper/2605.18005/) (2026)

## Abstract (arXiv)

Predict-then-Optimize is a framework for using machine learning to perform decision-making under uncertainty. The central research question it asks is, "How can the structure of a decision-making task be used to tailor ML models for that specific task?" To this end, recent work has proposed learning task-specific loss functions that capture this underlying structure. However, current approaches make restrictive assumptions about the form of these losses and their impact on ML model behavior. These assumptions both lead to approaches with high computational cost, and when they are violated in practice, poor performance. In this paper, we propose solutions to these issues, avoiding the aforementioned assumptions and utilizing the ML model's features to increase the sample efficiency of learning loss functions. We empirically show that our method achieves state-of-the-art results in four domains from the literature, often requiring an order of magnitude fewer samples than comparable methods from past work. Moreover, our approach outperforms the best existing method by nearly 200% when the localness assumption is broken.