# Eval4DiRec: A Unified and Systematic Evaluation Framework for Diffusion-based Recommender Systems

arXiv: https://arxiv.org/abs/2609.34404 · HTML: https://mlsanity.com/paper/2609.34404/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion recommenders, one of 14 directions in Diffusion models.
- Builds on 58 of its 107 references that are in the map.
- First posted on arXiv 28 Sep 2026.

## Placed in

- Diffusion models / [Diffusion recommenders](https://mlsanity.com/diffusion/diffusion-recommenders/)

## Builds on (most cited first, 30 of 58)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation](https://mlsanity.com/paper/2002.02126/) (2020)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Image-based Recommendations on Styles and Substitutes](https://mlsanity.com/paper/1506.04757/) (2015)
- [Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering](https://mlsanity.com/paper/1602.01585/) (2016)
- [Understanding Diffusion Models: A Unified Perspective](https://mlsanity.com/paper/2208.11970/) (2022)
- [A Survey on Generative Diffusion Model](https://mlsanity.com/paper/2209.02646/) (2022)
- [RecBole: Towards a Unified, Comprehensive and Efficient Framework for Recommendation Algorithms](https://mlsanity.com/paper/2011.01731/) (2020)
- [STAN: Spatio-Temporal Attention Network for Next Location Recommendation](https://mlsanity.com/paper/2102.04095/) (2021)
- [DiffuRec: A Diffusion Model for Sequential Recommendation](https://mlsanity.com/paper/2304.00686/) (2023)
- [Diffusion Recommender Model](https://mlsanity.com/paper/2304.04971/) (2023)
- [Denoising Implicit Feedback for Recommendation](https://mlsanity.com/paper/2006.04153/) (2020)
- [Generate What You Prefer: Reshaping Sequential Recommendation via Guided Diffusion](https://mlsanity.com/paper/2310.20453/) (2023)
- [A Review of Modern Recommender Systems Using Generative Models (Gen-RecSys)](https://mlsanity.com/paper/2404.00579/) (2024)
- [Bootstrap Latent Representations for Multi-modal Recommendation](https://mlsanity.com/paper/2207.05969/) (2022)
- [Diffusion Models for Non-autoregressive Text Generation: A Survey](https://mlsanity.com/paper/2303.06574/) (2023)
- [DiffKG: Knowledge Graph Diffusion Model for Recommendation](https://mlsanity.com/paper/2312.16890/) (2023)
- [Improved Noise Schedule for Diffusion Training](https://mlsanity.com/paper/2407.03297/) (2024)
- [Diffusion Augmentation for Sequential Recommendation](https://mlsanity.com/paper/2309.12858/) (2023)
- [Evaluation of Session-based Recommendation Algorithms](https://mlsanity.com/paper/1803.09587/) (2018)
- [Sequential Recommendation via Stochastic Self-Attention](https://mlsanity.com/paper/2201.06035/) (2022)
- [Plug-in Diffusion Model for Sequential Recommendation](https://mlsanity.com/paper/2401.02913/) (2024)
- [Denoising Diffusion Recommender Model](https://mlsanity.com/paper/2401.06982/) (2024)
- [Graph Signal Diffusion Model for Collaborative Filtering](https://mlsanity.com/paper/2311.08744/) (2023)
- [A Diffusion model for POI recommendation](https://mlsanity.com/paper/2304.07041/) (2023)
- [Conditional Denoising Diffusion for Sequential Recommendation](https://mlsanity.com/paper/2304.11433/) (2023)
- [DiffMM: Multi-Modal Diffusion Model for Recommendation](https://mlsanity.com/paper/2406.11781/) (2024)
- [RecDiff: Diffusion Model for Social Recommendation](https://mlsanity.com/paper/2406.01629/) (2024)

## Built on by (most cited first, 1 of 1)

- [HEM-Med: A Hierarchical Agentic Memory Framework for Safe Multi-Center Medication Recommendation](https://mlsanity.com/paper/2610.20191/) (2026)

## Abstract (arXiv)

Leveraging the strong generative capabilities and stable training dynamics of diffusion models, diffusion-based recommender systems (RSs) have recently emerged as a novel recommendation paradigm, attracting increasing attention from both academia and industry. However, despite the rapid growth of diffusion-based RSs, a critical issue has emerged: the lack of a unified and systematic quantitative evaluation benchmark, which often results in irreproducible experimental results and unfair comparisons across studies due to inconsistent data processing, training configurations, inference procedures, and evaluation protocols. To address this challenge, we propose Eval4DiRec, the first unified and open-source evaluation framework specifically designed for diffusion-based RSs. Eval4DiRec supports 14 representative diffusion-based RS models across five different recommendation scenarios, providing consistent and reproducible experimental settings to systematically assess their performance. Built upon this framework, we conduct extensive empirical studies to benchmark these models under unified protocols. The results highlight the strong potential of diffusion models for recommendation while also revealing key factors and practical challenges that substantially affect their performance, thereby establishing a solid foundation to facilitate fair evaluation and guide future research in this promising field. Our code and data are available at: https://github.com/wangcong2001/Eval4DiRec.