# Aligning Target-Aware Molecule Diffusion Models with Exact Energy Optimization

arXiv: https://arxiv.org/abs/2407.01648 · HTML: https://mlsanity.com/paper/2407.01648/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Placed in Flow matching, time series and science, one of 14 directions in Diffusion models.
- Builds on 26 of its 44 references that are in the map.
- First posted on arXiv 1 Jul 2024.

## Placed in

- Diffusion models / [Flow matching, time series and science](https://mlsanity.com/diffusion/flow-matching-time-series-and-science/)

## Builds on (most cited first, 26 of 26)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [Structured Denoising Diffusion Models in Discrete State-Spaces](https://mlsanity.com/paper/2107.03006/) (2021)
- [A General Theoretical Paradigm to Understand Learning from Human Preferences](https://mlsanity.com/paper/2310.12036/) (2023)
- [Diffusion Model Alignment Using Direct Preference Optimization](https://mlsanity.com/paper/2311.12908/) (2023)
- [Junction Tree Variational Autoencoder for Molecular Graph Generation](https://mlsanity.com/paper/1802.04364/) (2018)
- [GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generation](https://mlsanity.com/paper/2203.02923/) (2022)
- [GraphAF: a Flow-based Autoregressive Model for Molecular Graph Generation](https://mlsanity.com/paper/2001.09382/) (2020)
- [Grammar Variational Autoencoder](https://mlsanity.com/paper/1703.01925/) (2017)
- [Pocket2Mol: Efficient Molecular Sampling Based on 3D Protein Pockets](https://mlsanity.com/paper/2205.07249/) (2022)
- [3D Equivariant Diffusion for Target-Aware Molecule Generation and Affinity Prediction](https://mlsanity.com/paper/2303.03543/) (2023)
- [Generalized Preference Optimization: A Unified Approach to Offline Alignment](https://mlsanity.com/paper/2402.05749/) (2024)
- [A 3D Generative Model for Structure-Based Drug Design](https://mlsanity.com/paper/2203.10446/) (2022)
- [Fine-Tuning of Continuous-Time Diffusion Models as Entropy-Regularized Control](https://mlsanity.com/paper/2402.15194/) (2024)
- [Generating 3D Molecules for Target Protein Binding](https://mlsanity.com/paper/2204.09410/) (2022)
- [DecompDiff: Diffusion Models with Decomposed Priors for Structure-Based Drug Design](https://mlsanity.com/paper/2403.07902/) (2024)
- [Noise Contrastive Alignment of Language Models with Explicit Rewards](https://mlsanity.com/paper/2402.05369/) (2024)
- [Generating 3D Molecules Conditional on Receptor Binding Sites with Deep Generative Models](https://mlsanity.com/paper/2110.15200/) (2021)
- [Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedback](https://mlsanity.com/paper/1707.07402/) (2017)
- [Feedback Efficient Online Fine-Tuning of Diffusion Models](https://mlsanity.com/paper/2402.16359/) (2024)
- [State-specific protein-ligand complex structure prediction with a multi-scale deep generative model](https://mlsanity.com/paper/2209.15171/) (2022)
- [Antigen-Specific Antibody Design via Direct Energy-based Preference Optimization](https://mlsanity.com/paper/2403.16576/) (2024)
- [Learning Subpocket Prototypes for Generalizable Structure-based Drug Design](https://mlsanity.com/paper/2305.13997/) (2023)

## Built on by (most cited first, 25 of 25)

- [Alignment of Diffusion Models: Fundamentals, Challenges, and Future](https://mlsanity.com/paper/2409.07253/) (2024)
- [Multi-domain Distribution Learning for De Novo Drug Design](https://mlsanity.com/paper/2508.17815/) (2025)
- [Decomposed Direct Preference Optimization for Structure-Based Drug Design](https://mlsanity.com/paper/2407.13981/) (2024)
- [Prior-Guided Flow Matching for Target-Aware Molecule Design with Learnable Atom Number](https://mlsanity.com/paper/2509.01486/) (2025)
- [Sharpe Ratio-Guided Active Learning for Preference Optimization in RLHF](https://mlsanity.com/paper/2503.22137/) (2025)
- [De Novo Molecular Design Enabled by Direct Preference Optimization and Curriculum Learning](https://mlsanity.com/paper/2504.01389/) (2025)
- [Aligning Protein Conformation Ensemble Generation with Physical Feedback](https://mlsanity.com/paper/2505.24203/) (2025)
- [Unraveling the Potential of Diffusion Models in Small Molecule Generation](https://mlsanity.com/paper/2507.08005/) (2025)
- [Controllable 3D Molecular Generation for Structure-Based Drug Design Through Bayesian Flow Networks and Gradient Integration](https://mlsanity.com/paper/2508.21468/) (2025)
- [MolChord: Structure-Sequence Alignment for Protein-Guided Drug Design](https://mlsanity.com/paper/2510.27671/) (2025)
- [Empowering LLMs for Structure-Based Drug Design via Exploration-Augmented Latent Inference](https://mlsanity.com/paper/2601.15333/) (2026)
- [MolPIF: A Parameter Interpolation Flow Model for Molecule Generation](https://mlsanity.com/paper/2507.13762/) (2025)
- [Generating Developable 3D Molecules via Pocket-Conditioned Diffusion and Property-Aware Optimization](https://mlsanity.com/paper/2607.12349/) (2026)
- [Fragment-Masked Diffusion for Molecular Optimization](https://mlsanity.com/paper/2408.09106/) (2024)
- [TransDiffSBDD: Causality-Aware Multi-Modal Structure-Based Drug Design](https://mlsanity.com/paper/2503.20913/) (2025)
- [Fine-tuning Pocket-Aware Diffusion Models via Denoising Policy Optimization](https://mlsanity.com/paper/2605.17693/) (2026)
- [Fine-Tuning Diffusion Models for Molecular Generation via Reinforcement Learning and Fast Sampling](https://mlsanity.com/paper/2606.01220/) (2026)
- [CORAL: Learning Amyloid Fibril Ligand Docking with Cooperative Binding Rewards](https://mlsanity.com/paper/2607.17412/) (2026)
- [Do Language Models Dream of Binding Molecules? Benchmarking LLMs under Spatial Constraints](https://mlsanity.com/paper/2607.18144/) (2026)
- [Fused Bayesian Flow Networks for Dual-Target Molecular Design](https://mlsanity.com/paper/2608.01007/) (2026)
- [SurfSpec: Enhancing Off-Target-Agnostic Specificity by Bounding Pocket-Ligand Geometric Mismatch](https://mlsanity.com/paper/2609.02963/) (2026)
- [PocketVE: Stable and Property-Guided Structure-Based Drug Design with Variance-Exploding Diffusion](https://mlsanity.com/paper/2609.08101/) (2026)
- [A Systematic Evaluation of Molecule Generation Models for De Novo Drug Design: From Benchmarks to Practical Insights](https://mlsanity.com/paper/2609.10099/) (2026)
- [FestDPO: Few-step Generator Alignment with Direct Preference Optimization](https://mlsanity.com/paper/2609.34673/) (2026)
- [RIDE: Reference-Anchored Inference-Time Diffusion Editing for Scaffold Hopping](https://mlsanity.com/paper/2609.35623/) (2026)

## Abstract (arXiv)

Generating ligand molecules for specific protein targets, known as structure-based drug design, is a fundamental problem in therapeutics development and biological discovery. Recently, target-aware generative models, especially diffusion models, have shown great promise in modeling protein-ligand interactions and generating candidate drugs. However, existing models primarily focus on learning the chemical distribution of all drug candidates, which lacks effective steerability on the chemical quality of model generations. In this paper, we propose a novel and general alignment framework to align pretrained target diffusion models with preferred functional properties, named AliDiff. AliDiff shifts the target-conditioned chemical distribution towards regions with higher binding affinity and structural rationality, specified by user-defined reward functions, via the preference optimization approach. To avoid the overfitting problem in common preference optimization objectives, we further develop an improved Exact Energy Preference Optimization method to yield an exact and efficient alignment of the diffusion models, and provide the closed-form expression for the converged distribution. Empirical studies on the CrossDocked2020 benchmark show that AliDiff can generate molecules with state-of-the-art binding energies with up to -7.07 Avg. Vina Score, while maintaining strong molecular properties. Code is available at https://github.com/MinkaiXu/AliDiff.