# GAS: Generative Auto-bidding with Post-training Search

arXiv: https://arxiv.org/abs/2412.17018 · HTML: https://mlsanity.com/paper/2412.17018/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 47 references that are in the map.
- First posted on arXiv 22 Dec 2024.

## Builds on (most cited first, 22 of 22)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Conservative Q-Learning for Offline Reinforcement Learning](https://mlsanity.com/paper/2006.04779/) (2020)
- [Decision Transformer: Reinforcement Learning via Sequence Modeling](https://mlsanity.com/paper/2106.01345/) (2021)
- [Offline Reinforcement Learning with Implicit Q-Learning](https://mlsanity.com/paper/2110.06169/) (2021)
- [Off-Policy Deep Reinforcement Learning without Exploration](https://mlsanity.com/paper/1812.02900/) (2018)
- [SimPO: Simple Preference Optimization with a Reference-Free Reward](https://mlsanity.com/paper/2405.14734/) (2024)
- [Is Conditional Generative Modeling all you need for Decision-Making?](https://mlsanity.com/paper/2211.15657/) (2022)
- [Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision](https://mlsanity.com/paper/2312.09390/) (2023)
- [Tutorial on Variational Autoencoders](https://mlsanity.com/paper/1606.05908/) (2016)
- [Disentangling Length from Quality in Direct Preference Optimization](https://mlsanity.com/paper/2403.19159/) (2024)
- [Online Convex Optimization with Stochastic Constraints](https://mlsanity.com/paper/1708.03741/) (2017)
- [Better Training of GFlowNets with Local Credit and Incomplete Trajectories](https://mlsanity.com/paper/2302.01687/) (2023)
- [Cascade Reward Sampling for Efficient Decoding-Time Alignment](https://mlsanity.com/paper/2406.16306/) (2024)
- [Softmax Deep Double Deterministic Policy Gradients](https://mlsanity.com/paper/2010.09177/) (2020)
- [Sustainable Online Reinforcement Learning for Auto-bidding](https://mlsanity.com/paper/2210.07006/) (2022)
- [AuctionNet: A Novel Benchmark for Decision-Making in Large-Scale Games](https://mlsanity.com/paper/2412.10798/) (2024)
- [A Cooperative-Competitive Multi-Agent Framework for Auto-bidding in Online Advertising](https://mlsanity.com/paper/2106.06224/) (2021)
- [Robust Auction Design in the Auto-bidding World](https://mlsanity.com/paper/2111.02468/) (2021)
- [Towards Efficient Auctions in an Auto-bidding World](https://mlsanity.com/paper/2103.13356/) (2021)
- [Sequential Recommendation for Optimizing Both Immediate Feedback and Long-term Retention](https://mlsanity.com/paper/2404.03637/) (2024)

## Built on by (most cited first, 6 of 6)

- [Generative Auto-Bidding in Large-Scale Competitive Auctions via Diffusion Completer-Aligner](https://mlsanity.com/paper/2509.03348/) (2025)
- [VAO: Validation-Aligned Optimization for Cross-Task Generative Auto-Bidding](https://mlsanity.com/paper/2510.07760/) (2025)
- [Q-Regularized Generative Auto-Bidding: From Suboptimal Trajectories to Optimal Policies](https://mlsanity.com/paper/2601.02754/) (2026)
- [JD-BP: A Joint-Decision Generative Framework for Auto-Bidding and Pricing](https://mlsanity.com/paper/2604.05845/) (2026)
- [DRIVE: Distributional and Retrieval-Augmented Bidding with Value Evaluation](https://mlsanity.com/paper/2606.14192/) (2026)
- [AIGB-R1: Self-Evolving Generative Auto-Bidding via Hierarchical Planner-Executor Optimization](https://mlsanity.com/paper/2607.17281/) (2026)

## Abstract (arXiv)

Auto-bidding is essential in facilitating online advertising by automatically placing bids on behalf of advertisers. Generative auto-bidding, which generates bids based on an adjustable condition using models like transformers and diffusers, has recently emerged as a new trend due to its potential to learn optimal strategies directly from data and adjust flexibly to preferences. However, generative models suffer from low-quality data leading to a mismatch between the condition, like return to go, and true action value, especially in long sequential decision-making. Besides, the majority preference in the dataset may hinder models' generalization ability on minority advertisers' preferences. While it is possible to collect high-quality data and retrain multiple models for different preferences, the high cost makes it unaffordable, hindering the advancement of auto-bidding into the era of large foundation models. To address this, we propose a flexible and practical Generative Auto-bidding scheme using post-training Search, termed GAS, to refine a base policy model's output and adapt to various preferences. We use weak-to-strong search alignment by training small critics for different preferences and an MCTS-inspired search to refine the model's output. Specifically, a novel voting mechanism with transformer-based critics trained with policy indications could enhance search alignment performance. Additionally, utilizing the search, we provide a fine-tuning method for high-frequency preference scenarios considering computational efficiency. Extensive experiments conducted on the real-world dataset and online A/B test on the Kuaishou advertising platform demonstrate the effectiveness of GAS, achieving significant improvements, e.g., 4.60% increment of target cost.