# ProphetNet-Ads: A Looking Ahead Strategy for Generative Retrieval Models in Sponsored Search Engine

arXiv: https://arxiv.org/abs/2010.10789 · HTML: https://mlsanity.com/paper/2010.10789/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Oct 2020.

## Built on by (most cited first, 11 of 11)

- [Break the Sequential Dependency of LLM Inference Using Lookahead Decoding](https://mlsanity.com/paper/2402.02057/) (2024)
- [Diversity driven Query Rewriting in Search Advertising](https://mlsanity.com/paper/2106.03816/) (2021)
- [Unified Generative & Dense Retrieval for Query Rewriting in Sponsored Search](https://mlsanity.com/paper/2209.05861/) (2022)
- [VALUE: Value-Aware Large Language Model for Query Rewriting via Weighted Trie in Sponsored Search](https://mlsanity.com/paper/2504.05321/) (2025)
- [Scaling the Vocabulary of Non-autoregressive Models for Efficient Generative Retrieval](https://mlsanity.com/paper/2406.06739/) (2024)
- [Improving Retrieval in Sponsored Search by Leveraging Query Context Signals](https://mlsanity.com/paper/2407.14346/) (2024)
- [Modified Query Expansion Through Generative Adversarial Networks for Information Extraction in E-Commerce](https://mlsanity.com/paper/2301.00036/) (2022)
- [BEAR: Towards Beam-Search-Aware Optimization for Recommendation with Large Language Models](https://mlsanity.com/paper/2601.22925/) (2026)
- [Quotient Space-Based Keyword Retrieval in Sponsored Search](https://mlsanity.com/paper/2105.12371/) (2021)
- [Lost in Decoding? Reproducing and Stress-Testing the Look-Ahead Prior in Generative Retrieval](https://mlsanity.com/paper/2604.23396/) (2026)
- [Next-Gen Sponsored Search: Crafting the Perfect Query with Inventory-Aware RAG (InvAwr-RAG) Based GenAI](https://mlsanity.com/paper/2607.03880/) (2026)

## Abstract (arXiv)

In a sponsored search engine, generative retrieval models are recently proposed to mine relevant advertisement keywords for users' input queries. Generative retrieval models generate outputs token by token on a path of the target library prefix tree (Trie), which guarantees all of the generated outputs are legal and covered by the target library. In actual use, we found several typical problems caused by Trie-constrained searching length. In this paper, we analyze these problems and propose a looking ahead strategy for generative retrieval models named ProphetNet-Ads. ProphetNet-Ads improves the retrieval ability by directly optimizing the Trie-constrained searching space. We build a dataset from a real-word sponsored search engine and carry out experiments to analyze different generative retrieval models. Compared with Trie-based LSTM generative retrieval model proposed recently, our single model result and integrated result improve the recall by 15.58% and 18.8% respectively with beam size 5. Case studies further demonstrate how these problems are alleviated by ProphetNet-Ads clearly.