# An Embedding Learning Framework for Numerical Features in CTR Prediction

arXiv: https://arxiv.org/abs/2012.08986 · HTML: https://mlsanity.com/paper/2012.08986/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Dec 2020.

## Built on by (most cited first, 30 of 37)

- [On Embeddings for Numerical Features in Tabular Deep Learning](https://mlsanity.com/paper/2203.05556/) (2022)
- [ExcelFormer: A neural network surpassing GBDTs on tabular data](https://mlsanity.com/paper/2301.02819/) (2023)
- [Click-Through Rate Prediction in Online Advertising: A Literature Review](https://mlsanity.com/paper/2202.10462/) (2022)
- [Embedding in Recommender Systems: A Survey](https://mlsanity.com/paper/2310.18608/) (2023)
- [Tele-Knowledge Pre-training for Fault Analysis](https://mlsanity.com/paper/2210.11298/) (2022)
- [Alleviating Cold-start Problem in CTR Prediction with A Variational Embedding Learning Framework](https://mlsanity.com/paper/2201.10980/) (2022)
- [Ads Recommendation in a Collapsed and Entangled World](https://mlsanity.com/paper/2403.00793/) (2024)
- [Memorize, Factorize, or be Na"ive: Learning Optimal Feature Interaction Methods for CTR Prediction](https://mlsanity.com/paper/2108.01265/) (2021)
- [Adaptive Low-Precision Training for Embeddings in Click-Through Rate Prediction](https://mlsanity.com/paper/2212.05735/) (2022)
- [COPR: Consistency-Oriented Pre-Ranking for Online Advertising](https://mlsanity.com/paper/2306.03516/) (2023)
- [ERASE: Benchmarking Feature Selection Methods for Deep Recommender Systems](https://mlsanity.com/paper/2403.12660/) (2024)
- [OptDist: Learning Optimal Distribution for Customer Lifetime Value Prediction](https://mlsanity.com/paper/2408.08585/) (2024)
- [Deep Feature Embedding for Tabular Data](https://mlsanity.com/paper/2408.17162/) (2024)
- [Learning Enhanced Representations for Tabular Data via Neighborhood Propagation](https://mlsanity.com/paper/2206.06587/) (2022)
- [Multi-Objective Personalized Product Retrieval in Taobao Search](https://mlsanity.com/paper/2210.04170/) (2022)
- [A Comprehensive Survey on Automated Machine Learning for Recommendations](https://mlsanity.com/paper/2204.01390/) (2022)
- [Addressing Cold-start Problem in Click-Through Rate Prediction via Supervised Diffusion Modeling](https://mlsanity.com/paper/2504.06270/) (2025)
- [Burn After Reading: Online Adaptation for Cross-domain Streaming Data](https://mlsanity.com/paper/2112.04345/) (2021)
- [Toward An Optimal Selection of Dialogue Strategies: A Target-Driven Approach for Intelligent Outbound Robots](https://mlsanity.com/paper/2206.10953/) (2022)
- [From Principle to Practice: Vertical Data Minimization for Machine Learning](https://mlsanity.com/paper/2311.10500/) (2023)
- [Exploiting Graph Structured Cross-Domain Representation for Multi-Domain Recommendation](https://mlsanity.com/paper/2302.05990/) (2023)
- [(GG) MoE vs. MLP on Tabular Data](https://mlsanity.com/paper/2502.03608/) (2025)
- [Enhancing Interpretability and Effectiveness in Recommendation with Numerical Features via Learning to Contrast the Counterfactual samples](https://mlsanity.com/paper/2509.03187/) (2025)
- [AutoAssign+: Automatic Shared Embedding Assignment in Streaming Recommendation](https://mlsanity.com/paper/2308.06965/) (2023)
- [Feature Distribution Shift Mitigation with Contrastive Pretraining for Intrusion Detection](https://mlsanity.com/paper/2404.15382/) (2024)
- [Retrieval-Oriented Knowledge for Click-Through Rate Prediction](https://mlsanity.com/paper/2404.18304/) (2024)
- [Given Users Recommendations Based on Reviews on Yelp](https://mlsanity.com/paper/2112.01762/) (2021)
- [Item Cold Start Recommendation via Adversarial Variational Auto-encoder Warm-up](https://mlsanity.com/paper/2302.14395/) (2023)
- [Enhancing Multi-field B2B Cloud Solution Matching via Contrastive Pre-training](https://mlsanity.com/paper/2402.07076/) (2024)
- [Cognitive Evolutionary Learning to Select Feature Interactions for Recommender Systems](https://mlsanity.com/paper/2405.18708/) (2024)

## Abstract (arXiv)

Click-Through Rate (CTR) prediction is critical for industrial recommender systems, where most deep CTR models follow an Embedding & Feature Interaction paradigm. However, the majority of methods focus on designing network architectures to better capture feature interactions while the feature embedding, especially for numerical features, has been overlooked. Existing approaches for numerical features are difficult to capture informative knowledge because of the low capacity or hard discretization based on the offline expertise feature engineering. In this paper, we propose a novel embedding learning framework for numerical features in CTR prediction (AutoDis) with high model capacity, end-to-end training and unique representation properties preserved. AutoDis consists of three core components: meta-embeddings, automatic discretization and aggregation. Specifically, we propose meta-embeddings for each numerical field to learn global knowledge from the perspective of field with a manageable number of parameters. Then the differentiable automatic discretization performs soft discretization and captures the correlations between the numerical features and meta-embeddings. Finally, distinctive and informative embeddings are learned via an aggregation function. Comprehensive experiments on two public and one industrial datasets are conducted to validate the effectiveness of AutoDis. Moreover, AutoDis has been deployed onto a mainstream advertising platform, where online A/B test demonstrates the improvement over the base model by 2.1% and 2.7% in terms of CTR and eCPM, respectively. In addition, the code of our framework is publicly available in MindSpore(https://gitee.com/mindspore/mindspore/tree/master/modelzoo/research/recommend/autodis).