mlsanityData to 7 Oct 2026

Learning to Allocate Incentives for Incentivized Advertising via Offline Model-Based Reinforcement Learning

arXiv 2608.28065 ↗First posted 28 Aug 2026cs.AIcited by 0

On the map: Offline RL and imitation learning from data → Decision transformers and sequence modeling

Abstract (arXiv)

Complete your ad view and grab a 5-cent bonus! In incentivized advertising, a platform promises users a bonus before observing downstream ad revenue, encouraging them to click and complete ads. It must balance the incentive promised in advance against the revenue realized afterward: insufficient incentives forfeit monetization opportunities, whereas excessive incentives reduce net profit. Because current incentives may also shape user expectations and future engagement, incentive allocation is a sequential decision problem with delayed revenue, cost sensitivity, and carryover effects. Existing work has not studied decision-making algorithms for this setting. Auto-bidding assumes available ad opportunities, while targeted promotion optimizes incentives outside the ad monetization pipeline. We formulate the problem as an MDP and develop an offline model-based RL framework for cost-controllable sequential incentive allocation. It learns a world model of user feedback and ad revenue, then performs conservative policy optimization. An independent counterfactual scorer evaluates each learned policy on held-out logs, enabling pre-launch selection without costly online exposure. Experiments on large-scale industrial data and online A/B tests show that the scorer provides a stable offline signal. The deployment path from causal inference to offline RL and then Offline-MBRL further validates the framework: MB-IQL improves per-user net profit by 7.96% over TD3+BC, whereas reverting to plain IQL reduces it by 6.56% (both (p<0.0001)).

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Decision transformers and sequence modeling, one of 13 directions in Offline RL and imitation learning from data. Builds on 24 of its 34 references that are in the map. First posted on arXiv 28 Aug 2026.

Builds on

24 of 34 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022BCRLSP: An Offline Reinforcement Learning Framework for Sequential Targeted Promotion16 Jul
  2. 2021Decision Transformer: Reinforcement Learning via Sequence Modeling2 Jun

Other references · 22

  1. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  2. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  3. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  4. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 882
  5. 2020MOPO: Model-based Offline Policy Optimizationcited by 630
  6. 2015Estimation and Inference of Heterogeneous Treatment Effects using Random Forestscited by 597
  7. 2019Introduction to Multi-Armed Banditscited by 475
  8. 2021COMBO: Conservative Offline Model-Based Policy Optimizationcited by 365
  9. 2019Batch Policy Learning under Constraintscited by 242
  10. 2018Top-K Off-Policy Correction for a REINFORCE Recommender Systemcited by 181
  11. 2018Deep Reinforcement Learning for Page-wise Recommendationscited by 99
  12. 2017Real-Time Bidding by Reinforcement Learning in Display Advertisingcited by 90
10 more
  1. 2024Is Value Learning Really the Main Bottleneck in Offline RL?cited by 82
  2. 2018Budget Constrained Bidding by Model-free Reinforcement Learning in Display Advertisingcited by 60
  3. 2025OneTrans: Unified Feature Interaction and Sequence Modeling with One Transformer in Industrial Recommendercited by 34
  4. 2019A Unified Framework for Marketing Budget Allocationcited by 33
  5. 2024Model-based Offline Reinforcement Learning with Lower Expectile Q-Learningcited by 17
  6. 2020Spending Money Wisely: Online Electronic Coupon Allocation based on Real-Time User Intent Detectioncited by 13
  7. 2019Cost-Effective Incentive Allocation via Structured Counterfactual Inferencecited by 12
  8. 2024End-to-End Cost-Effective Incentive Recommendation under Budget Constraint with Uplift Modelingcited by 10
  9. 2022A Framework for Multi-stage Bonus Allocation in meal delivery Platformcited by 4
  10. 2016Revenue Maximization in Incentivized Social Advertisingcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Offline Reinforcement Learning for Optimizing Production Bidding Policies
  2. 2025Multi-task Offline Reinforcement Learning for Online Advertising in Recommender Systems
  3. 2026DARA: Few-shot Budget Allocation in Online Advertising via In-Context Decision Making with RL-Finetuned LLMs
  4. 2026OneBid: A Unified Auto-Bidding Foundation Model for Diverse oCPX Advertising Scenarios
  5. 2026AHBid: An Adaptable Hierarchical Bidding Framework for Cross-Channel Advertising
  6. 2025Generative Large-Scale Pre-trained Models for Automated Ad Bidding Optimization
  7. 2026DRIVE: Distributional and Retrieval-Augmented Bidding with Value Evaluation
  8. 2025Enhancing Generative Auto-bidding with Offline Reward Evaluation and Policy Search