# Truthful Aggregation of LLMs with an Application to Online Advertising

arXiv: https://arxiv.org/abs/2405.05905 · HTML: https://mlsanity.com/paper/2405.05905/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 41 references that are in the map.
- First posted on arXiv 9 May 2024.

## Builds on (most cited first, 12 of 12)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [A General Theoretical Paradigm to Understand Learning from Human Preferences](https://mlsanity.com/paper/2310.12036/) (2023)
- [Data Selection for Language Models via Importance Resampling](https://mlsanity.com/paper/2302.03169/) (2023)
- [Generative Social Choice](https://mlsanity.com/paper/2309.01291/) (2023)
- [Auctions with LLM Summaries](https://mlsanity.com/paper/2404.08126/) (2024)
- [Online Advertisements with LLMs: Opportunities and Challenges](https://mlsanity.com/paper/2311.07601/) (2023)
- [Q-Probe: A Lightweight Approach to Reward Maximization for Language Models](https://mlsanity.com/paper/2402.14688/) (2024)

## Built on by (most cited first, 29 of 29)

- [Ad Auctions for LLMs via Retrieval Augmented Generation](https://mlsanity.com/paper/2406.09459/) (2024)
- [Game Theory Meets Large Language Models: A Systematic Survey with Taxonomy and New Frontiers](https://mlsanity.com/paper/2502.09053/) (2025)
- [LLM-Auction: Generative Auction towards LLM-Native Advertising](https://mlsanity.com/paper/2512.10551/) (2025)
- [Online Learning from Strategic Human Feedback in LLM Fine-Tuning](https://mlsanity.com/paper/2412.16834/) (2024)
- [LLM-Powered Preference Elicitation in Combinatorial Assignment](https://mlsanity.com/paper/2502.10308/) (2025)
- [Ad Insertion in LLM-Generated Responses](https://mlsanity.com/paper/2601.19435/) (2026)
- [The Consensus Trap: Rescuing Multi-Agent LLMs from Adversarial Majorities via Token-Level Collaboration](https://mlsanity.com/paper/2604.17139/) (2026)
- [Effectively Steer LLM To Follow Preference via Building Confident Directions](https://mlsanity.com/paper/2503.02989/) (2025)
- [Strategyproof Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2503.09561/) (2025)
- [Beyond RLHF and NLHF: Population-Proportional Alignment under an Axiomatic Framework](https://mlsanity.com/paper/2506.05619/) (2025)
- [Think Before Writing: Feature-Level Multi-Objective Optimization for Generative Citation Visibility](https://mlsanity.com/paper/2604.19113/) (2026)
- [Prices, Bids, Values: One ML-Powered Combinatorial Auction to Rule Them All](https://mlsanity.com/paper/2411.09355/) (2024)
- [NaiAD: Initiate Data-Driven Research for LLM Advertising](https://mlsanity.com/paper/2605.09918/) (2026)
- [LERA: LLM-Enhanced RAG for Ad Auction in Generative Chatbots](https://mlsanity.com/paper/2605.16474/) (2026)
- [PILA: Plug-and-Play Insertion for LLM-native Advertising](https://mlsanity.com/paper/2607.25590/) (2026)
- [The Battling Influencers Game: Nash Equilibria Structure of a Potential Game and Implications to Value Alignment](https://mlsanity.com/paper/2502.01127/) (2025)
- [Learning Truthful Mechanisms without Discretization](https://mlsanity.com/paper/2506.22911/) (2025)
- [TeamCMU at Touch'e: Adversarial Co-Evolution for Advertisement Integration and Detection in Conversational Search](https://mlsanity.com/paper/2507.00509/) (2025)
- [Sponsored Questions and How to Auction Them](https://mlsanity.com/paper/2512.03975/) (2025)
- [Incentive-Aware Multi-Fidelity Optimization for Generative Advertising in Large Language Models](https://mlsanity.com/paper/2604.06263/) (2026)
- [Generative AI Advertising as a Problem of Trustworthy Commercial Intervention](https://mlsanity.com/paper/2605.18673/) (2026)
- [Evaluating and Pricing Advertisements in AI-Generated Responses](https://mlsanity.com/paper/2607.27686/) (2026)
- [Detecting RAG Advertisements Across Advertising Styles](https://mlsanity.com/paper/2603.04925/) (2026)
- [TourMart: A Parametric Audit Instrument for Commission Steering in LLM Travel Agents](https://mlsanity.com/paper/2605.10440/) (2026)
- [Truthful Online Preference Aggregation for LLM Fine-Tuning in Mobile Crowdsourcing](https://mlsanity.com/paper/2605.24052/) (2026)
- [Decentralized Aggregation of LLM Predictions via Wagering Mechanisms](https://mlsanity.com/paper/2607.04389/) (2026)
- [LLM-OSDA: An Optimal-Stopping Dynamic Auction for Native Advertising in Multi-Turn LLM Conversations](https://mlsanity.com/paper/2608.00123/) (2026)
- [AdsWorldEngine: A Self-Evolving Conversational Advertising Agent through Orchestrator and Tool Coevolution](https://mlsanity.com/paper/2608.13833/) (2026)
- [LLMAdBench: A Human Preference Benchmark for Advertising in LLM Responses](https://mlsanity.com/paper/2609.32533/) (2026)

## Abstract (arXiv)

The next frontier of online advertising is revenue generation from LLM-generated content. We consider a setting where advertisers aim to influence the responses of an LLM to align with their interests, while platforms seek to maximize advertiser value and ensure user satisfaction. The challenge is that advertisers' preferences generally conflict with those of the user, and advertisers may misreport their preferences. To address this, we introduce MOSAIC, an auction mechanism that ensures that truthful reporting is a dominant strategy for advertisers and that aligns the utility of each advertiser with their contribution to social welfare. Importantly, the mechanism operates without LLM fine-tuning or access to model weights and provably converges to the output of the optimally fine-tuned LLM as computational resources increase. Additionally, it can incorporate contextual information about advertisers, which significantly improves social welfare. Through experiments with a publicly available LLM, we show that MOSAIC leads to high advertiser value and platform revenue with low computational overhead. While our motivating application is online advertising, our mechanism can be applied in any setting with monetary transfers, making it a general-purpose solution for truthfully aggregating the preferences of self-interested agents over LLM-generated replies.