mlsanityData to 14 Oct 2026

LLMDistill4Ads: Using Cross-Encoders to Distill from LLM Signals for Advertiser Keyphrase Recommendations at eBay

arXiv 2508.03628 ↗First posted 5 Aug 2025cs.IR cs.AI cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

E-commerce sellers are advised to bid on keyphrases to boost their advertising campaigns. These keyphrases must be relevant to prevent irrelevant items from cluttering Search systems and to maintain positive seller perception. It is vital that keyphrase suggestions align with seller, Search, and buyer judgments. Given the challenges in collecting negative feedback in these systems, LLMs have been used as a scalable proxy for human judgments. We present an empirical study on a major e-commerce platform of a distillation framework involving an LLM teacher, a cross-encoder assistant and a bi-encoder Embedding Based Retrieval (EBR) student model, aimed at mitigating click-induced biases and provide more diverse keyphrase recommendations while aligning advertising, search and buyer preferences.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 59 of its 94 references that are in the map. First posted on arXiv 5 Aug 2025.

Builds on

59 of 94 references.

References · 59

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  6. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  7. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  8. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  9. 2024A Survey on LLM-as-a-Judgecited by 1,423
  10. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,216
  11. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
  12. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 849
38 more
  1. 2019Improved Knowledge Distillation via Teacher Assistantcited by 573
  2. 2024Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inferencecited by 554
  3. 2020MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devicescited by 482
  4. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 476
  5. 2020DeeBERT: Dynamic Early Exiting for Accelerating BERT Inferencecited by 332
  6. 2020Bias and Debias in Recommender System: A Survey and Future Directionscited by 240
  7. 2017Efficient Natural Language Response Suggestion for Smart Replycited by 229
  8. 2021RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-rankingcited by 176
  9. 2016Unbiased Learning-to-Rank with Biased Feedbackcited by 157
  10. 2020Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillationcited by 145
  11. 2020Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Taskscited by 138
  12. 2021Comparing Kullback-Leibler Divergence and Mean Squared Error Loss in Knowledge Distillationcited by 134
  13. 2023Large language models can accurately predict searcher preferencescited by 131
  14. 2017Deep Keyphrase Generationcited by 102
  15. 2024An Empirical Study of LLM-as-a-Judge for LLM Evaluation: Fine-tuned Judge Model is not a General Substitute for GPT-4cited by 94
  16. 2018One Size Does Not Fit All: Generating and Evaluating Variable Number of Keyphrasescited by 52
  17. 2021NeuralNDCG: Direct Optimisation of a Ranking Metric via Differentiable Relaxation of Sortingcited by 41
  18. 2022ERNIE-Search: Bridging Cross-Encoder with Dual-Encoder via Self On-the-fly Distillation for Dense Passage Retrievalcited by 41
  19. 2024GISTEmbed: Guided In-sample Selection of Training Negatives for Text Embedding Fine-tuningcited by 40
  20. 2021DeepXML: A Deep Extreme Multi-Label Learning Framework Applied to Short Text Documentscited by 37
  21. 2018Title-Guided Encoding for Keyphrase Generationcited by 36
  22. 2024Trans-Tokenization and Cross-lingual Vocabulary Transfers: Language Adaptation of LLMs for Low-Resource NLPcited by 36
  23. 2022NGAME: Negative Mining-aware Mini-batching for Extreme Classificationcited by 32
  24. 2020Is Retriever Merely an Approximator of Reader?cited by 30
  25. 2019An Integrated Approach for Keyphrase Generation via Exploring the Power of Retrieval and Extractioncited by 28
  26. 2024Large Language Models as Recommender Systems: A Study of Popularity Biascited by 26
  27. 2024Don't Use LLMs to Make Relevance Judgmentscited by 26
  28. 2022PROD: Progressive Distillation for Dense Retrievalcited by 21
  29. 2022On Modality Bias Recognition and Reductioncited by 20
  30. 2025DRAMA: Diverse Augmentation from Large Language Models to Smaller Dense Retrieverscited by 19
  31. 2022Open Vocabulary Extreme Classification Using Generative Modelscited by 13
  32. 2022WR-ONE2SET: Towards Well-Calibrated Keyphrase Generationcited by 13
  33. 2023Rec4Ad: A Free Lunch to Mitigate Sample Selection Bias for Ads CTR Prediction in Taobaocited by 11
  34. 2024Piccolo2: General Text Embedding with Multi-task Hybrid Loss Trainingcited by 10
  35. 2024D2LLM: Decomposed and Distilled Large Language Models for Semantic Searchcited by 9
  36. 2024Improving Pinterest Search Relevance Using Large Language Modelscited by 8
  37. 2023An Offline Metric for the Debiasedness of Click Modelscited by 6
  38. 2024Graphite: A Graph-based Extreme Multi-Label Short Text Classifier for Keyphrase Recommendationcited by 6

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Scaling Up Efficient Small Language Models Serving and Deployment for Semantic Job Searchcited by 3
  2. 2026AdNanny: One Reasoning LLM for All Offline Ads Recommendation Taskscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025OMS: On-the-fly, Multi-Objective, Self-Reflective Ad Keyword Generation via LLM Agent
  2. 2024OKG: On-the-Fly Keyword Generation in Sponsored Search Advertising
  3. 2026LLM-based User-Product Relevance Evaluation for Dynamic Product Advertising
  4. 2025LLMs for Customized Marketing Content Generation and Evaluation at Scale
  5. 2026LLMAdBench: A Human Preference Benchmark for Advertising in LLM Responses
  6. 2026NaiAD: Initiate Data-Driven Research for LLM Advertising
  7. 2026MMRM: A Multiplex Multimodal Representation Model for Product Ranking in E-commerce Search
  8. 2025Rewrite-to-Rank: Optimizing Ad Visibility via Retrieval-Aware Text Rewriting