# PDB-Struct: A Comprehensive Benchmark for Structure-based Protein Design

arXiv: https://arxiv.org/abs/2312.00080 · HTML: https://mlsanity.com/paper/2312.00080/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 53 references that are in the map.
- First posted on arXiv 30 Nov 2023.

## Builds on (most cited first, 9 of 9)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Learning from Protein Structure with Geometric Vector Perceptrons](https://mlsanity.com/paper/2009.01411/) (2020)
- [Diffusion probabilistic modeling of protein backbones in 3D for the motif-scaffolding problem](https://mlsanity.com/paper/2206.04119/) (2022)
- [Pre-training via Denoising for Molecular Property Prediction](https://mlsanity.com/paper/2206.00133/) (2022)
- [PiFold: Toward effective and efficient protein inverse folding](https://mlsanity.com/paper/2209.12643/) (2022)
- [Structure-informed Language Models Are Protein Designers](https://mlsanity.com/paper/2302.01649/) (2023)
- [Graph Denoising Diffusion for Inverse Protein Folding](https://mlsanity.com/paper/2306.16819/) (2023)
- [Protein Sequence and Structure Co-Design with Equivariant Translation](https://mlsanity.com/paper/2210.08761/) (2022)
- [Knowledge-Design: Pushing the Limit of Protein Design via Knowledge Refinement](https://mlsanity.com/paper/2305.15151/) (2023)

## Built on by (most cited first, 6 of 6)

- [ProteinBench: A Holistic Evaluation of Protein Foundation Models](https://mlsanity.com/paper/2409.06744/) (2024)
- [Bridge-IF: Learning Inverse Protein Folding with Markov Bridges](https://mlsanity.com/paper/2411.02120/) (2024)
- [ProteinZero: Self-Improving Protein Generation via Online Reinforcement Learning](https://mlsanity.com/paper/2506.07459/) (2025)
- [PQA: Zero-shot Protein Question Answering for Free-form Scientific Enquiry with Large Language Models](https://mlsanity.com/paper/2402.13653/) (2024)
- [Multi-state Protein Design with DynamicMPNN](https://mlsanity.com/paper/2507.21938/) (2025)
- [SurfDesign: Effective Protein Design on Molecular Surfaces](https://mlsanity.com/paper/2606.07567/) (2026)

## Abstract (arXiv)

Structure-based protein design has attracted increasing interest, with numerous methods being introduced in recent years. However, a universally accepted method for evaluation has not been established, since the wet-lab validation can be overly time-consuming for the development of new algorithms, and the textitin silico validation with recovery and perplexity metrics is efficient but may not precisely reflect true foldability. To address this gap, we introduce two novel metrics: refoldability-based metric, which leverages high-accuracy protein structure prediction models as a proxy for wet lab experiments, and stability-based metric, which assesses whether models can assign high likelihoods to experimentally stable proteins. We curate datasets from high-quality CATH protein data, high-throughput textitde novo designed proteins, and mega-scale experimental mutagenesis experiments, and in doing so, present the PDB-Struct benchmark that evaluates both recent and previously uncompared protein design methods. Experimental results indicate that ByProt, ProteinMPNN, and ESM-IF perform exceptionally well on our benchmark, while ESM-Design and AF-Design fall short on the refoldability metric. We also show that while some methods exhibit high sequence recovery, they do not perform as well on our new benchmark. Our proposed benchmark paves the way for a fair and comprehensive evaluation of protein design methods in the future. Code is available at https://github.com/WANG-CR/PDB-Struct.