# Evaluating and Pricing Advertisements in AI-Generated Responses

arXiv: https://arxiv.org/abs/2607.27686 · HTML: https://mlsanity.com/paper/2607.27686/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 36 references that are in the map.
- First posted on arXiv 30 Jul 2026.

## Builds on (most cited first, 12 of 12)

- [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://mlsanity.com/paper/2005.04118/) (2020)
- [Wide & Deep Learning for Recommender Systems](https://mlsanity.com/paper/1606.07792/) (2016)
- [DeepFM: A Factorization-Machine based Neural Network for CTR Prediction](https://mlsanity.com/paper/1703.04247/) (2017)
- [Deep Interest Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1706.06978/) (2017)
- [Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations](https://mlsanity.com/paper/2110.05719/) (2021)
- [Squared Earth Mover's Distance-based Loss for Training Deep Neural Networks](https://mlsanity.com/paper/1611.05916/) (2016)
- [Truthful Aggregation of LLMs with an Application to Online Advertising](https://mlsanity.com/paper/2405.05905/) (2024)
- [LLM-Auction: Generative Auction towards LLM-Native Advertising](https://mlsanity.com/paper/2512.10551/) (2025)
- [Ad Insertion in LLM-Generated Responses](https://mlsanity.com/paper/2601.19435/) (2026)
- [GEM-Bench: A Benchmark for Ad-Injected Response Generation within Generative Engine Marketing](https://mlsanity.com/paper/2509.14221/) (2025)
- [PILA: Plug-and-Play Insertion for LLM-native Advertising](https://mlsanity.com/paper/2607.25590/) (2026)
- [LLM-HYPER: Generative CTR Modeling for Cold-Start Ad Personalization via LLM-Based Hypernetworks](https://mlsanity.com/paper/2604.12096/) (2026)

## Built on by (most cited first, 1 of 1)

- [LLMAdBench: A Human Preference Benchmark for Advertising in LLM Responses](https://mlsanity.com/paper/2609.32533/) (2026)

## Abstract (arXiv)

As search increasingly shifts toward LLM-driven answer engines, advertising is becoming embedded within the generated response itself and should therefore be evaluated for both user utility and commercial value. The key challenge is click-through intent: behavioural logs are unavailable, human annotation resists calibration, and frontier LLM judges conflate intent with linguistic fluency. These gaps compound, as principled pricing presupposes a continuous intent signal, while generating such a signal presupposes supervision that is currently unavailable. We construct the missing supervision through a psychologically grounded agent simulation framework, and distil it into a parameter-efficient evaluator that predicts click-through intent, together with the three companion dimensions of ad quality, as smooth, differentiable estimates. Validated through sign-certain behavioural perturbations, the evaluator surpasses frontier zero-shot judges on relevance sensitivity (79% versus 60-67%), tracks graded content degradation, generalises without error to 103 fictional products, and agrees with human preference in 86% of pairwise judgements across five annotators, with agreement rising in the evaluator's confidence. Upon its estimates we build the pricing layer directly, deriving the unique payment rule under which truthful bidding is optimal, demonstrating it on a best-of-k allocation, and extending the mechanism to non-monotone allocations. The same differentiable signal stands ready as a training objective for ad generation.