# Metric-Optimized Example Weights

arXiv: https://arxiv.org/abs/1805.10582 · HTML: https://mlsanity.com/paper/1805.10582/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 May 2018.

## Built on by (most cited first, 8 of 8)

- [FORML: Learning to Reweight Data for Fairness](https://mlsanity.com/paper/2202.01719/) (2022)
- [MetricOpt: Learning to Optimize Black-Box Evaluation Metrics](https://mlsanity.com/paper/2104.10631/) (2021)
- [Perspectives on Incorporating Expert Feedback into Model Updates](https://mlsanity.com/paper/2205.06905/) (2022)
- [Optimizing Black-box Metrics with Iterative Example Weighting](https://mlsanity.com/paper/2102.09492/) (2021)
- [Regularization Penalty Optimization for Addressing Data Quality Variance in OoD Algorithms](https://mlsanity.com/paper/2206.05749/) (2022)
- [The Power of Few: Accelerating and Enhancing Data Reweighting with Coreset Selection](https://mlsanity.com/paper/2403.12166/) (2024)
- [The Benefits and Risks of Transductive Approaches for AI Fairness](https://mlsanity.com/paper/2406.12011/) (2024)
- [An Efficient Plugin Method for Metric Optimization of Black-Box Models](https://mlsanity.com/paper/2503.02119/) (2025)

## Abstract (arXiv)

Real-world machine learning applications often have complex test metrics, and may have training and test data that are not identically distributed. Motivated by known connections between complex test metrics and cost-weighted learning, we propose addressing these issues by using a weighted loss function with a standard loss, where the weights on the training examples are learned to optimize the test metric on a validation set. These metric-optimized example weights can be learned for any test metric, including black box and customized ones for specific applications. We illustrate the performance of the proposed method on diverse public benchmark datasets and real-world applications. We also provide a generalization bound for the method.