# Generative Auto-Bidding with Unified Modeling and Exploration

arXiv: https://arxiv.org/abs/2605.19457 · HTML: https://mlsanity.com/paper/2605.19457/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 42 references that are in the map.
- First posted on arXiv 19 May 2026.

## Builds on (most cited first, 22 of 22)

- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Conservative Q-Learning for Offline Reinforcement Learning](https://mlsanity.com/paper/2006.04779/) (2020)
- [Offline Reinforcement Learning with Implicit Q-Learning](https://mlsanity.com/paper/2110.06169/) (2021)
- [A Minimalist Approach to Offline Reinforcement Learning](https://mlsanity.com/paper/2106.06860/) (2021)
- [Benchmarking Batch Deep Reinforcement Learning Algorithms](https://mlsanity.com/paper/1910.01708/) (2019)
- [Online Convex Optimization with Stochastic Constraints](https://mlsanity.com/paper/1708.03741/) (2017)
- [Real-Time Bidding by Reinforcement Learning in Display Advertising](https://mlsanity.com/paper/1701.02490/) (2017)
- [MADiff: Offline Multi-agent Learning with Diffusion Models](https://mlsanity.com/paper/2305.17330/) (2023)
- [Real-Time Bidding with Multi-Agent Reinforcement Learning in Display Advertising](https://mlsanity.com/paper/1802.09756/) (2018)
- [What Makes a Good Diffusion Planner for Decision Making?](https://mlsanity.com/paper/2503.00535/) (2025)
- [Bid Optimization by Multivariable Control in Display Advertising](https://mlsanity.com/paper/1905.10928/) (2019)
- [Real-time Bidding for Online Advertising: Measurement and Analysis](https://mlsanity.com/paper/1306.6542/) (2013)
- [Sustainable Online Reinforcement Learning for Auto-bidding](https://mlsanity.com/paper/2210.07006/) (2022)
- [AuctionNet: A Novel Benchmark for Decision-Making in Large-Scale Games](https://mlsanity.com/paper/2412.10798/) (2024)
- [Generative Auto-Bidding with Value-Guided Explorations](https://mlsanity.com/paper/2504.14587/) (2025)
- [Feedback Control of Real-Time Display Advertising](https://mlsanity.com/paper/1603.01055/) (2016)
- [A Cooperative-Competitive Multi-Agent Framework for Auto-bidding in Online Advertising](https://mlsanity.com/paper/2106.06224/) (2021)
- [RTBAgent: A LLM-based Agent System for Real-Time Bidding](https://mlsanity.com/paper/2502.00792/) (2025)
- [Trajectory-wise Iterative Reinforcement Learning Framework for Auto-bidding](https://mlsanity.com/paper/2402.15102/) (2024)
- [Optimal Return-to-Go Guided Decision Transformer for Auto-Bidding in Advertisement](https://mlsanity.com/paper/2506.21956/) (2025)
- [Expert-Guided Diffusion Planner for Auto-Bidding](https://mlsanity.com/paper/2508.08687/) (2025)
- [Bid2X: Revealing Dynamics of Bidding Environment in Online Advertising from A Foundation Model Lens](https://mlsanity.com/paper/2510.23410/) (2025)

## Abstract (arXiv)

Automated bidding is central to modern digital advertising. Early rule-based methods lacked adaptability, while subsequent Reinforcement Learning approaches modeled bidding as a Markov Decision Process but struggled with long-term dependencies. Recent generative models show promise, yet they lack explicit mechanisms to balance exploration and safety, relying solely on action perturbations or trajectory guidance without a safety fallback. This results in inefficient exploration and elevated financial risk for advertising platforms. To address this gap, we propose GUIDE (Generative Auto-Bidding with Unified Modeling and Exploration), a framework that synergistically integrates directed exploration with a safe fallback mechanism. GUIDE employs a Decision Transformer (DT) to jointly model historical bidding actions and environmental state transitions. A Q-value module guides the DT's exploration via regularization constraints, while an Inverse Dynamics Module (IDM) leverages DT-predicted future states to infer robust, behaviorally consistent actions as a safe policy fallback. The Q-value module then adaptively selects the final action between these two options, balancing exploration and safety. Together, these components form an integrated "explore-safeguard-select" pipeline that unifies efficiency and safety. We conduct extensive experiments on public datasets, in simulated auction environments, and through large-scale online deployment on Taobao, a leading Chinese advertising platform. Results show GUIDE consistently outperforms state-of-the-art baselines across all scenarios. In real-world deployment, GUIDE achieves notable gains: +4.10% ad GMV, +1.40% ad clicks, +1.66% ad cost, and +3.52% ad ROI, demonstrating its effectiveness and strong industrial applicability.