mlsanityData to 14 Oct 2026

AIGB-R1: Self-Evolving Generative Auto-Bidding via Hierarchical Planner-Executor Optimization

arXiv 2607.17281 ↗First posted 19 Jul 2026cs.LG cs.AIcited by 0

On the map: Synthetic data and self-improvement for LLMs → Self-evolving LLM agents

Abstract (arXiv)

Auto-bidding plays an essential role in online advertising, automatically adjusting bids for advertisers to optimize their commercial goals. The emerging AI-Generated Bidding (AIGB) paradigm widely adopts generative modeling to optimize bidding strategies, yet suffers from the limited mode coverage of offline datasets and inadequate task-state understanding, hindering effective exploration of optimal strategies. Large Language Models (LLMs), with prior world knowledge and reasoning capabilities, offer a promising approach to overcome these limitations. However, directly applying LLMs to auto-bidding tasks faces inherent challenges in limited numerical precision, hallucinations, and inference latency. To address these limitations, we propose AIGB-R1, a hierarchical self-evolving auto-bidding framework aiming to enhance AI-Generated Bidding via LLMs' Reasoning capabilities, comprising a high-level Planner module for macro-level strategy planning and a low-level Executor module for fine-grained decision-making. Building upon this, we design an experience-driven self-evolving loop, enabling autonomous strategy exploration and optimization from accumulated experience. We adopt a two-stage pipeline of offline pre-training and post-training alignment, and build an interactive bidding simulation environment for strategy rollout. Furthermore, we propose Decoupled Group Relative Policy Optimization (D-GRPO) to achieve end-to-end optimization via advantage decoupling. Experimental results on a large-scale public dataset demonstrate the effectiveness of AIGB-R1.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Self-evolving LLM agents, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 34 of its 65 references that are in the map. First posted on arXiv 19 Jul 2026.

Builds on

34 of 65 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning4 Nov

Other references · 33

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  3. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  4. 2023Qwen Technical Reportcited by 3,344
  5. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  6. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  7. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  8. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,241
  9. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 963
  10. 2023ExpeL: LLM Agents Are Experiential Learnerscited by 876
  11. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 840
  12. 2022Is Conditional Generative Modeling all you need for Decision-Making?cited by 516
21 more
  1. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 500
  2. 2025Group-in-Group Policy Optimization for LLM Agent Trainingcited by 497
  3. 2024Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translationcited by 394
  4. 2024KTO: Model Alignment as Prospect Theoretic Optimizationcited by 264
  5. 2022Prompting Decision Transformer for Few-Shot Policy Generalizationcited by 146
  6. 2017Online Convex Optimization with Stochastic Constraintscited by 95
  7. 2016Display Advertising with Real-Time Bidding (RTB) and Behavioural Targetingcited by 42
  8. 2022Sustainable Online Reinforcement Learning for Auto-biddingcited by 29
  9. 2025Generative Auto-Bidding with Value-Guided Explorationscited by 23
  10. 2021A Cooperative-Competitive Multi-Agent Framework for Auto-bidding in Online Advertisingcited by 22
  11. 2021Robust Auction Design in the Auto-bidding Worldcited by 21
  12. 2021Towards Efficient Auctions in an Auto-bidding Worldcited by 19
  13. 2024From Words to Actions: Unveiling the Theoretical Underpinnings of LLM-Driven Autonomous Systemscited by 17
  14. 2024Trajectory-wise Iterative Reinforcement Learning Framework for Auto-biddingcited by 9
  15. 2024GAS: Generative Auto-bidding with Post-training Searchcited by 6
  16. 2025Think Twice, Act Once: A Co-Evolution Framework of LLM and RL for Large-Scale Decision Makingcited by 5
  17. 2025Generative Auto-Bidding in Large-Scale Competitive Auctions via Diffusion Completer-Alignercited by 5
  18. 2026DARA: Few-shot Budget Allocation in Online Advertising via In-Context Decision Making with RL-Finetuned LLMscited by 4
  19. 2025Generative Large-Scale Pre-trained Models for Automated Ad Bidding Optimizationcited by 3
  20. 2025Generative Bid Shading in Real-Time Bidding Advertisingcited by 1
  21. 2026LBM: Hierarchical Large Auto-Bidding Model via Reasoning and Actingcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024AIGB: Generative Auto-bidding via Conditional Diffusion Modeling
  2. 2026Strategy-Aware Parameter-Efficient Adaptation for LLM-based Auto-Bidding ↗
  3. 2026On the Role of Language Representations in Auto-Bidding: Findings and Implications
  4. 2025Enhancing Generative Auto-bidding with Offline Reward Evaluation and Policy Search
  5. 2026Generative Auto-Bidding with Unified Modeling and Exploration
  6. 2026AHBid: An Adaptable Hierarchical Bidding Framework for Cross-Channel Advertising
  7. 2025Expert-Guided Diffusion Planner for Auto-Bidding
  8. 2026JD-BP: A Joint-Decision Generative Framework for Auto-Bidding and Pricing