Bridging the Information Gap: Semantic Densification and Hindsight Distillation for Cold-Start Prediction
Not on the map. Why?
Abstract (arXiv)
New-user cold-start is a critical bottleneck for e-commerce platforms: predicting user lifetime value (LTV) and conversion rate (CVR) for users with sparse interaction history. Two prior directions -- LLM-based semantic augmentation and learning using privileged information (LUPI) -- each face a key limitation. First, LLM augmentation produces unstructured rationales that are noisy and hard to operationalize in production. Second, naive student-teacher distillation can be brittle due to an information gap between the privileged teacher and the sparse student; moreover, this gap is heterogeneous across users. We propose SemRaD, a Semantic Reasoning-aware Distillation framework addressing both limitations. First, a Structured Semantic Reasoning Pipeline replaces free-form rationales with a structured schema built via a discover-curate-audit workflow, producing per user a Densified Semantic Profile (consumed by the deployed student via a Semantic-Gated Encoder that focuses on the most informative dimensions) and a Hindsight Distillation Target reconciled from pre- and post-conversion reasoning (used only at training). Second, to bridge this gap and handle its heterogeneity, a Hindsight-Aware Distillation Network transfers privileged knowledge via the hindsight target, with Distillation Experts improving transfer under per-user variability. On a large-scale industrial dataset, SemRaD lifts +1.9% LTV (Gini) and +1.0% CVR (AUROC) over a production-grade base; a four-week online A/B at Keeta confirms +1.0% LTV / +0.43% CVR. SemRaD also matches the production system's LTV using only 9% of the training data while improving CVR by 0.8%.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 13 of its 25 references that are in the map. First posted on arXiv 19 Jul 2026.
Builds on
13 of 25 references.
References · 13
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
- 2014FitNets: Hints for Thin Deep Netscited by 1,619
- 2018Self-Attentive Sequential Recommendationcited by 1,169
- 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 778
- 2017DeepFM: A Factorization-Machine based Neural Network for CTR Predictioncited by 673
- 2017Deep Interest Network for Click-Through Rate Predictioncited by 541
- 2017Deep & Cross Network for Ad Click Predictionscited by 392
- 2023TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendationcited by 336
- 2015Unifying distillation and privileged informationcited by 218
- 2018Entire Space Multi-Task Model: An Effective Approach for Estimating Post-Click Conversion Ratecited by 172
- 2024Leveraging LLM Reasoning Enhances Personalized Recommender Systemscited by 26
Built on by
Not cited yet.