# What is the Right Notion of Distance between Predict-then-Optimize Tasks?

arXiv: https://arxiv.org/abs/2409.06997 · HTML: https://mlsanity.com/paper/2409.06997/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 46 references that are in the map.
- First posted on arXiv 11 Sep 2024.

## Builds on (most cited first, 26 of 26)

- [OptNet: Differentiable Optimization as a Layer in Neural Networks](https://mlsanity.com/paper/1703.00443/) (2017)
- [Differentiable Convex Optimization Layers](https://mlsanity.com/paper/1910.12430/) (2019)
- [Smart "Predict, then Optimize"](https://mlsanity.com/paper/1710.08005/) (2017)
- [Joint Distribution Optimal Transportation for Domain Adaptation](https://mlsanity.com/paper/1705.08848/) (2017)
- [Melding the Data-Decisions Pipeline: Decision-Focused Learning for Combinatorial Optimization](https://mlsanity.com/paper/1809.05504/) (2018)
- [Geometric Dataset Distances via Optimal Transport](https://mlsanity.com/paper/2002.02923/) (2020)
- [Transferability and Hardness of Supervised Classification Tasks](https://mlsanity.com/paper/1908.08142/) (2019)
- [A Wasserstein-type distance in the space of Gaussian Mixture Models](https://mlsanity.com/paper/1907.05254/) (2019)
- [LAVA: Data Valuation without Pre-Specified Learning Algorithms](https://mlsanity.com/paper/2305.00054/) (2023)
- [OpenDataVal: a Unified Benchmark for Data Valuation](https://mlsanity.com/paper/2306.10577/) (2023)
- [Hierarchical Optimal Transport for Document Representation](https://mlsanity.com/paper/1906.10827/) (2019)
- [A Principled Approach for Learning Task Similarity in Multitask Learning](https://mlsanity.com/paper/1903.09109/) (2019)
- [PyEPO: A PyTorch-based End-to-End Predict-then-Optimize Library for Linear and Integer Programming](https://mlsanity.com/paper/2206.14234/) (2022)
- [Structured Optimal Transport](https://mlsanity.com/paper/1712.06199/) (2017)
- [Active learning for data streams: a survey](https://mlsanity.com/paper/2302.08893/) (2023)
- [Automatically Learning Compact Quality-aware Surrogates for Optimization Problems](https://mlsanity.com/paper/2006.10815/) (2020)
- [Generalizing Point Embeddings using the Wasserstein Space of Elliptical Distributions](https://mlsanity.com/paper/1805.07594/) (2018)
- [Estimate-Then-Optimize versus Integrated-Estimation-Optimization versus Sample Average Approximation: A Stochastic Dominance Perspective](https://mlsanity.com/paper/2304.06833/) (2023)
- [Wasserstein regularization for sparse multi-task regression](https://mlsanity.com/paper/1805.07833/) (2018)
- [Leaving the Nest: Going Beyond Local Loss Functions for Predict-Then-Optimize](https://mlsanity.com/paper/2305.16830/) (2023)
- [Differentiable Top-k Operator with Optimal Transport](https://mlsanity.com/paper/2002.06504/) (2020)
- [Learning Embeddings into Entropic Wasserstein Spaces](https://mlsanity.com/paper/1905.03329/) (2019)
- [Decision-Focused Learning without Differentiable Optimization: Learning Locally Optimized Decision Losses](https://mlsanity.com/paper/2203.16067/) (2022)
- [TaskMet: Task-Driven Metric Learning for Model Learning](https://mlsanity.com/paper/2312.05250/) (2023)
- [Active Learning For Contextual Linear Optimization: A Margin-Based Approach](https://mlsanity.com/paper/2305.06584/) (2023)
- [Decision-Focused Evaluation of Worst-Case Distribution Shift](https://mlsanity.com/paper/2407.03557/) (2024)

## Built on by (most cited first, 2 of 2)

- [Decision-Focused Learning: When and Why Traditional Prediction Models Fail](https://mlsanity.com/paper/2606.21773/) (2026)
- [ASSS: A Differentiable Adversarial Framework for Task-Aware Data Reduction](https://mlsanity.com/paper/2601.02081/) (2026)

## Abstract (arXiv)

Comparing datasets is a fundamental task in machine learning, essential for various learning paradigms-from evaluating train and test datasets for model generalization to using dataset similarity for detecting data drift. While traditional notions of dataset distances offer principled measures of similarity, their utility has largely been assessed through prediction error minimization. However, in Predict-then-Optimize (PtO) frameworks, where predictions serve as inputs for downstream optimization tasks, model performance is measured through decision regret rather than prediction error. In this work, we propose OTD3 (Optimal Transport Decision-aware Dataset Distance), a novel dataset distance that incorporates downstream decisions in addition to features and labels. We show that traditional feature-label distances lack informativeness in PtO settings, while OTD3 more effectively captures adaptation success. We also derive a PtO-specific adaptation bound based on this distance. Empirically, we show that our proposed distance accurately predicts model transferability across three different PtO tasks from the literature. The code is available at https://github.com/paularodr/OTD3.