# Contrastive losses as generalized models of global epistasis

arXiv: https://arxiv.org/abs/2305.03136 · HTML: https://mlsanity.com/paper/2305.03136/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 51 references that are in the map.
- First posted on arXiv 4 May 2023.

## Builds on (most cited first, 5 of 5)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Conditioning by adaptive sampling for robust design](https://mlsanity.com/paper/1901.10060/) (2019)
- [Tranception: protein fitness prediction with autoregressive transformers and inference-time retrieval](https://mlsanity.com/paper/2205.13760/) (2022)
- [Deep Extrapolation for Attribute-Enhanced Generation](https://mlsanity.com/paper/2107.02968/) (2021)
- [Spectral Regularization Allows Data-frugal Learning over Combinatorial Spaces](https://mlsanity.com/paper/2210.02604/) (2022)

## Built on by (most cited first, 2 of 2)

- [Metalic: Meta-Learning In-Context with Protein Language Models](https://mlsanity.com/paper/2410.08355/) (2024)
- [Low-N Protein Activity Optimization with FolDE](https://mlsanity.com/paper/2510.24053/) (2025)

## Abstract (arXiv)

Fitness functions map large combinatorial spaces of biological sequences to properties of interest. Inferring these multimodal functions from experimental data is a central task in modern protein engineering. Global epistasis models are an effective and physically-grounded class of models for estimating fitness functions from observed data. These models assume that a sparse latent function is transformed by a monotonic nonlinearity to emit measurable fitness. Here we demonstrate that minimizing supervised contrastive loss functions, such as the Bradley-Terry loss, is a simple and flexible technique for extracting the sparse latent function implied by global epistasis. We argue by way of a fitness-epistasis uncertainty principle that the nonlinearities in global epistasis models can produce observed fitness functions that do not admit sparse representations, and thus may be inefficient to learn from observations when using a Mean Squared Error (MSE) loss (a common practice). We show that contrastive losses are able to accurately estimate a ranking function from limited data even in regimes where MSE is ineffective and validate the practical utility of this insight by demonstrating that contrastive loss functions result in consistently improved performance on benchmark tasks.