# Learning Surrogates via Deep Embedding

arXiv: https://arxiv.org/abs/2007.00799 · HTML: https://mlsanity.com/paper/2007.00799/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jul 2020.

## Built on by (most cited first, 11 of 11)

- [What If We Only Use Real Datasets for Scene Text Recognition? Toward Scene Text Recognition With Fewer Labels](https://mlsanity.com/paper/2103.04400/) (2021)
- [Recall@k Surrogate Loss with Large Batches and Similarity Mixup](https://mlsanity.com/paper/2108.11179/) (2021)
- [Generalized Differentiable RANSAC](https://mlsanity.com/paper/2212.13185/) (2022)
- [MetricOpt: Learning to Optimize Black-Box Evaluation Metrics](https://mlsanity.com/paper/2104.10631/) (2021)
- [Amortized Synthesis of Constrained Configurations Using a Differentiable Surrogate](https://mlsanity.com/paper/2106.09019/) (2021)
- [FEDS -- Filtered Edit Distance Surrogate](https://mlsanity.com/paper/2103.04635/) (2021)
- [Relational Surrogate Loss Learning](https://mlsanity.com/paper/2202.13197/) (2022)
- [Differentiable Topology-Preserved Distance Transform for Pulmonary Airway Segmentation](https://mlsanity.com/paper/2209.08355/) (2022)
- [Searching Parameterized AP Loss for Object Detection](https://mlsanity.com/paper/2112.05138/) (2021)
- [Emulating Non-Differentiable Metrics via Knowledge-Guided Learning: Introducing the Minkowski Image Loss](https://mlsanity.com/paper/2604.11422/) (2026)
- [Robustly Invertible Nonlinear Dynamics and the BiLipREN: From Inversion-Based Control to Generative Trajectory Modelling](https://mlsanity.com/paper/2607.10026/) (2026)

## Abstract (arXiv)

This paper proposes a technique for training a neural network by minimizing a surrogate loss that approximates the target evaluation metric, which may be non-differentiable. The surrogate is learned via a deep embedding where the Euclidean distance between the prediction and the ground truth corresponds to the value of the evaluation metric. The effectiveness of the proposed technique is demonstrated in a post-tuning setup, where a trained model is tuned using the learned surrogate. Without a significant computational overhead and any bells and whistles, improvements are demonstrated on challenging and practical tasks of scene-text recognition and detection. In the recognition task, the model is tuned using a surrogate approximating the edit distance metric and achieves up to 39% relative improvement in the total edit distance. In the detection task, the surrogate approximates the intersection over union metric for rotated bounding boxes and yields up to 4.25% relative improvement in the F1 score.