# Deep Learning over Multi-field Categorical Data: A Case Study on User Response Prediction

arXiv: https://arxiv.org/abs/1601.02376 · HTML: https://mlsanity.com/paper/1601.02376/

## Key facts

- Cited by 67 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jan 2016.

## Built on by (most cited first, 30 of 67)

- [A Survey on Accuracy-oriented Neural Recommendation: From Collaborative Filtering to Information-rich Recommendation](https://mlsanity.com/paper/2104.13030/) (2021)
- [TALENT: A Tabular Analytics and Learning Toolbox](https://mlsanity.com/paper/2407.04057/) (2024)
- [End-to-End User Behavior Retrieval in Click-Through RatePrediction Model](https://mlsanity.com/paper/2108.04468/) (2021)
- [Representation Learning for Tabular Data: A Comprehensive Survey](https://mlsanity.com/paper/2504.16109/) (2025)
- [Deep Learning for Click-Through Rate Estimation](https://mlsanity.com/paper/2104.10584/) (2021)
- [FM2: Field-matrixed Factorization Machines for Recommender Systems](https://mlsanity.com/paper/2102.12994/) (2021)
- [Click-Through Rate Prediction in Online Advertising: A Literature Review](https://mlsanity.com/paper/2202.10462/) (2022)
- [On the Embedding Collapse when Scaling up Recommendation Models](https://mlsanity.com/paper/2310.04400/) (2023)
- [Embedding in Recommender Systems: A Survey](https://mlsanity.com/paper/2310.18608/) (2023)
- [Concept-Aware Denoising Graph Neural Network for Micro-Video Recommendation](https://mlsanity.com/paper/2109.13527/) (2021)
- [User Response Prediction in Online Advertising](https://mlsanity.com/paper/2101.02342/) (2021)
- [Learning Graph Meta Embeddings for Cold-Start Ads in Click-Through Rate Prediction](https://mlsanity.com/paper/2105.08909/) (2021)
- [GraphFM: Graph Factorization Machines for Feature Interaction Modeling](https://mlsanity.com/paper/2105.11866/) (2021)
- [ScaleFreeCTR: MixCache-based Distributed Training System for CTR Models with Huge Embedding Table](https://mlsanity.com/paper/2104.08542/) (2021)
- [Memorize, Factorize, or be Na"ive: Learning Optimal Feature Interaction Methods for CTR Prediction](https://mlsanity.com/paper/2108.01265/) (2021)
- [DHEN: A Deep and Hierarchical Ensemble Network for Large-Scale Click-Through Rate Prediction](https://mlsanity.com/paper/2203.11014/) (2022)
- [MvFS: Multi-view Feature Selection for Recommender System](https://mlsanity.com/paper/2309.02064/) (2023)
- [Causality-based CTR Prediction using Graph Neural Networks](https://mlsanity.com/paper/2301.12762/) (2023)
- [Unlocking the Transferability of Tokens in Deep Models for Tabular Data](https://mlsanity.com/paper/2310.15149/) (2023)
- [Multi-Scenario Ranking with Adaptive Feature Learning](https://mlsanity.com/paper/2306.16732/) (2023)
- [XCrossNet: Feature Structure-Oriented Learning for Click-Through Rate Prediction](https://mlsanity.com/paper/2104.10907/) (2021)
- [DCAP: Deep Cross Attentional Product Network for User Response Prediction](https://mlsanity.com/paper/2105.08649/) (2021)
- [Muddling Label Regularization: Deep Learning for Tabular Datasets](https://mlsanity.com/paper/2106.04462/) (2021)
- [Mixture of Virtual-Kernel Experts for Multi-Objective User Profile Modeling](https://mlsanity.com/paper/2106.07356/) (2021)
- [Boost CTR Prediction for New Advertisements via Modeling Visual Content](https://mlsanity.com/paper/2209.11727/) (2022)
- [Quaternion Factorization Machines: A Lightweight Solution to Intricate Feature Interaction Modelling](https://mlsanity.com/paper/2104.01716/) (2021)
- [Looking at CTR Prediction Again: Is Attention All You Need?](https://mlsanity.com/paper/2105.05563/) (2021)
- [CowClip: Reducing CTR Prediction Model Training Time from 12 hours to 10 minutes on 1 GPU](https://mlsanity.com/paper/2204.06240/) (2022)
- [TabGSL: Graph Structure Learning for Tabular Data Prediction](https://mlsanity.com/paper/2305.15843/) (2023)
- [Multi-Epoch Learning for Deep Click-Through Rate Prediction Models](https://mlsanity.com/paper/2305.19531/) (2023)

## Abstract (arXiv)

Predicting user responses, such as click-through rate and conversion rate, are critical in many web applications including web search, personalised recommendation, and online advertising. Different from continuous raw features that we usually found in the image and audio domains, the input features in web space are always of multi-field and are mostly discrete and categorical while their dependencies are little known. Major user response prediction models have to either limit themselves to linear models or require manually building up high-order combination features. The former loses the ability of exploring feature interactions, while the latter results in a heavy computation in the large feature space. To tackle the issue, we propose two novel models using deep neural networks (DNNs) to automatically learn effective patterns from categorical feature interactions and make predictions of users' ad clicks. To get our DNNs efficiently work, we propose to leverage three feature transformation methods, i.e., factorisation machines (FMs), restricted Boltzmann machines (RBMs) and denoising auto-encoders (DAEs). This paper presents the structure of our models and their efficient training algorithms. The large-scale experiments with real-world data demonstrate that our methods work better than major state-of-the-art models.