# Distribution-Aware End-to-End Embedding for Streaming Numerical Features in Click-Through Rate Prediction

arXiv: https://arxiv.org/abs/2602.03223 · HTML: https://mlsanity.com/paper/2602.03223/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 42 of its 71 references that are in the map.
- First posted on arXiv 3 Feb 2026.

## Builds on (most cited first, 30 of 42)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Wide & Deep Learning for Recommender Systems](https://mlsanity.com/paper/1606.07792/) (2016)
- [Revisiting Deep Learning Models for Tabular Data](https://mlsanity.com/paper/2106.11959/) (2021)
- [Deep & Cross Network for Ad Click Predictions](https://mlsanity.com/paper/1708.05123/) (2017)
- [Deep Interest Evolution Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1809.03672/) (2018)
- [Deep Learning Recommendation Model for Personalization and Recommendation Systems](https://mlsanity.com/paper/1906.00091/) (2019)
- [DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems](https://mlsanity.com/paper/2008.13535/) (2020)
- [AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks](https://mlsanity.com/paper/1810.11921/) (2018)
- [xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems](https://mlsanity.com/paper/1803.05170/) (2018)
- [On Embeddings for Numerical Features in Tabular Deep Learning](https://mlsanity.com/paper/2203.05556/) (2022)
- [Neural Factorization Machines for Sparse Predictive Analytics](https://mlsanity.com/paper/1708.05027/) (2017)
- [Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks](https://mlsanity.com/paper/1708.04617/) (2017)
- [Deep Learning over Multi-field Categorical Data: A Case Study on User Response Prediction](https://mlsanity.com/paper/1601.02376/) (2016)
- [Field-weighted Factorization Machines for Click-Through Rate Prediction in Display Advertising](https://mlsanity.com/paper/1806.03514/) (2018)
- [Adaptive Factorization Network: Learning Adaptive-Order Feature Interactions](https://mlsanity.com/paper/1909.03276/) (2019)
- [MaskNet: Introducing Feature-Wise Multiplication to CTR Ranking Models by Instance-Guided Mask](https://mlsanity.com/paper/2102.07619/) (2021)
- [Higher-Order Factorization Machines](https://mlsanity.com/paper/1607.07195/) (2016)
- [Product-based Neural Networks for User Response Prediction over Multi-field Categorical Data](https://mlsanity.com/paper/1807.00311/) (2018)
- [Neural Input Search for Large Scale Recommendation Models](https://mlsanity.com/paper/1907.04471/) (2019)
- [An Embedding Learning Framework for Numerical Features in CTR Prediction](https://mlsanity.com/paper/2012.08986/) (2020)
- [Towards Understanding the Overfitting Phenomenon of Deep Click-Through Rate Prediction Models](https://mlsanity.com/paper/2209.06053/) (2022)
- [Personalized Graph Signal Processing for Collaborative Filtering](https://mlsanity.com/paper/2302.02113/) (2023)
- [AgentCF++: Memory-enhanced LLM-based Agents for Popularity-aware Cross-domain Recommendations](https://mlsanity.com/paper/2502.13843/) (2025)
- [Network On Network for Tabular Data Classification in Real-world Applications](https://mlsanity.com/paper/2005.10114/) (2020)
- [Enhancing CTR Prediction with Context-Aware Feature Representation Learning](https://mlsanity.com/paper/2204.08758/) (2022)
- [AutoSeqRec: Autoencoder for Efficient Sequential Recommendation](https://mlsanity.com/paper/2308.06878/) (2023)
- [Ads Recommendation in a Collapsed and Entangled World](https://mlsanity.com/paper/2403.00793/) (2024)
- [Parameter-free Dynamic Graph Embedding for Link Prediction](https://mlsanity.com/paper/2210.08189/) (2022)
- [Triple Structural Information Modelling for Accurate, Explainable and Interactive Recommendation](https://mlsanity.com/paper/2304.11528/) (2023)
- [Computing Extremely Accurate Quantiles Using t-Digests](https://mlsanity.com/paper/1902.04023/) (2019)

## Abstract (arXiv)

This paper explores effective numerical feature embedding for Click-Through Rate prediction in streaming environments. Conventional static binning methods rely on offline statistics of numerical distributions; however, this inherently two-stage process often triggers semantic drift during bin boundary updates. While neural embedding methods enable end-to-end learning, they often discard explicit distributional information. Integrating such information end-to-end is challenging because streaming features often violate the i.i.d. assumption, precluding unbiased estimation of the population distribution via the expectation of order statistics. Furthermore, the critical context dependency of numerical distributions is often neglected. To this end, we propose DAES, an end-to-end framework designed to tackle numerical feature embedding in streaming training scenarios by integrating distributional information with an adaptive modulation mechanism. Specifically, we introduce an efficient reservoir-sampling-based distribution estimation method and two field-aware distribution modulation strategies to capture streaming distributions and field-dependent semantics. DAES significantly outperforms existing approaches as demonstrated by extensive offline and online experiments and has been fully deployed on a leading short-video platform with hundreds of millions of daily active users.