# Generative Modeling with Multi-Instance Reward Learning for E-commerce Creative Optimization

arXiv: https://arxiv.org/abs/2508.09730 · HTML: https://mlsanity.com/paper/2508.09730/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion and multimodal model alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 9 of its 18 references that are in the map.
- First posted on arXiv 13 Aug 2025.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Diffusion and multimodal model alignment](https://mlsanity.com/alignment/diffusion-and-multimodal-model-alignment/)

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep CTR Prediction in Display Advertising](https://mlsanity.com/paper/1609.06018/) (2016)
- [Learning to Create Better Ads: Generation and Ranking Approaches for Ad Creative Refinement](https://mlsanity.com/paper/2008.07467/) (2020)
- [Top-k Combinatorial Bandits with Full-Bandit Feedback](https://mlsanity.com/paper/1905.12624/) (2019)
- [A New Creative Generation Pipeline for Click-Through Rate with Stable Diffusion Model](https://mlsanity.com/paper/2401.10934/) (2024)
- [Striking Gold in Advertising: Standardization and Exploration of Ad Text Generation](https://mlsanity.com/paper/2309.12030/) (2023)
- [Automated Creative Optimization for E-Commerce Advertising](https://mlsanity.com/paper/2103.00436/) (2021)
- [Category-Specific CNN for Visual-aware CTR Prediction at JD.com](https://mlsanity.com/paper/2006.10337/) (2020)
- [Parallel Ranking of Ads and Creatives in Real-Time Advertising Systems](https://mlsanity.com/paper/2312.12750/) (2023)

## Built on by (most cited first, 1 of 1)

- [AgenticGen: Reward-Guided Agentic Video Generation for Advertising](https://mlsanity.com/paper/2609.09187/) (2026)

## Abstract (arXiv)

In e-commerce advertising, selecting the most compelling combination of creative elements -- such as titles, images, and highlights -- is critical for capturing user attention and driving conversions. However, existing methods often evaluate creative components individually, failing to navigate the exponentially large search space of possible combinations. To address this challenge, we propose a novel framework named GenCO that integrates generative modeling with multi-instance reward learning. Our unified two-stage architecture first employs a generative model to efficiently produce a diverse set of creative combinations. This generative process is optimized with reinforcement learning, enabling the model to effectively explore and refine its selections. Next, to overcome the challenge of sparse user feedback, a multi-instance learning model attributes combination-level rewards, such as clicks, to the individual creative elements. This allows the reward model to provide a more accurate feedback signal, which in turn guides the generative model toward creating more effective combinations. Deployed on a leading e-commerce platform, our approach has significantly increased advertising revenue, demonstrating its practical value. Additionally, we are releasing a large-scale industrial dataset to facilitate further research in this important domain.