# Kernel-Based Evaluation of Conditional Biological Sequence Models

arXiv: https://arxiv.org/abs/2510.15601 · HTML: https://mlsanity.com/paper/2510.15601/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 47 references that are in the map.
- First posted on arXiv 17 Oct 2025.

## Builds on (most cited first, 17 of 17)

- [Soft-DTW: a Differentiable Loss Function for Time-Series](https://mlsanity.com/paper/1703.01541/) (2017)
- [Evaluating model calibration in classification](https://mlsanity.com/paper/1902.06977/) (2019)
- [A Survey on Graph Kernels](https://mlsanity.com/paper/1903.11835/) (2019)
- [A Kernel Test of Goodness of Fit](https://mlsanity.com/paper/1602.02964/) (2016)
- [Measuring Sample Quality with Kernels](https://mlsanity.com/paper/1703.01717/) (2017)
- [A Measure-Theoretic Approach to Kernel Conditional Mean Embeddings](https://mlsanity.com/paper/2002.03689/) (2020)
- [PiFold: Toward effective and efficient protein inverse folding](https://mlsanity.com/paper/2209.12643/) (2022)
- [Learning Theory for Distribution Regression](https://mlsanity.com/paper/1411.2066/) (2014)
- [Efficient Aggregated Kernel Tests using Incomplete U-statistics](https://mlsanity.com/paper/2206.09194/) (2022)
- [Calibration tests beyond classification](https://mlsanity.com/paper/2210.13355/) (2022)
- [Testing Goodness of Fit of Conditional Density Models with Kernels](https://mlsanity.com/paper/2002.10271/) (2020)
- [Distribution Regression with Sliced Wasserstein Kernels](https://mlsanity.com/paper/2202.03926/) (2022)
- [Antibody-Antigen Docking and Design via Hierarchical Equivariant Refinement](https://mlsanity.com/paper/2207.06616/) (2022)
- [Compress Then Test: Powerful Kernel Testing in Near-linear Time](https://mlsanity.com/paper/2301.05974/) (2023)
- [A kernel Stein test of goodness of fit for sequential models](https://mlsanity.com/paper/2210.10741/) (2022)
- [Biological Sequence Kernels with Guaranteed Flexibility](https://mlsanity.com/paper/2304.03775/) (2023)
- [Fast and Scalable Score-Based Kernel Calibration Tests](https://mlsanity.com/paper/2510.14711/) (2025)

## Built on by (most cited first, 1 of 1)

- [Off-Manifold Collapse in Guided Protein Language Models](https://mlsanity.com/paper/2608.18597/) (2026)

## Abstract (arXiv)

We propose a set of kernel-based tools to evaluate the designs and tune the hyperparameters of conditional sequence models, with a focus on problems in computational biology. The backbone of our tools is a new measure of discrepancy between the true conditional distribution and the model's estimate, called the Augmented Conditional Maximum Mean Discrepancy (ACMMD). Provided that the model can be sampled from, the ACMMD can be estimated unbiasedly from data to quantify absolute model fit, integrated within hypothesis tests, and used to evaluate model reliability. We demonstrate the utility of our approach by analyzing a popular protein design model, ProteinMPNN. We are able to reject the hypothesis that ProteinMPNN fits its data for various protein families, and tune the model's temperature hyperparameter to achieve a better fit.