# Neural Factorization Machines for Sparse Predictive Analytics

arXiv: https://arxiv.org/abs/1708.05027 · HTML: https://mlsanity.com/paper/1708.05027/

## Key facts

- Cited by 183 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Aug 2017.

## Built on by (most cited first, 30 of 183)

- [Advances and Challenges in Conversational Recommender Systems: A Survey](https://mlsanity.com/paper/2101.09459/) (2021)
- [Recommendation Unlearning](https://mlsanity.com/paper/2201.06820/) (2022)
- [Knowledge Graph Contrastive Learning for Recommendation](https://mlsanity.com/paper/2205.00976/) (2022)
- [A Survey on Graph Representation Learning Methods](https://mlsanity.com/paper/2204.01855/) (2022)
- [A Survey on Accuracy-oriented Neural Recommendation: From Collaborative Filtering to Information-rich Recommendation](https://mlsanity.com/paper/2104.13030/) (2021)
- [RevCore: Review-augmented Conversational Recommendation](https://mlsanity.com/paper/2106.00957/) (2021)
- [Modeling the Sequential Dependence among Audience Multi-step Conversions with Multi-task Learning in Targeted Display Advertising](https://mlsanity.com/paper/2105.08489/) (2021)
- [PipAttack: Poisoning Federated Recommender Systems forManipulating Item Promotion](https://mlsanity.com/paper/2110.10926/) (2021)
- [Deep Learning for Click-Through Rate Estimation](https://mlsanity.com/paper/2104.10584/) (2021)
- [FM2: Field-matrixed Factorization Machines for Recommender Systems](https://mlsanity.com/paper/2102.12994/) (2021)
- [AutoLoss: Automated Loss Function Search in Recommendations](https://mlsanity.com/paper/2106.06713/) (2021)
- [Click-Through Rate Prediction in Online Advertising: A Literature Review](https://mlsanity.com/paper/2202.10462/) (2022)
- [AutoField: Automating Feature Selection in Deep Recommender Systems](https://mlsanity.com/paper/2204.09078/) (2022)
- [On the Embedding Collapse when Scaling up Recommendation Models](https://mlsanity.com/paper/2310.04400/) (2023)
- [Multiplex Behavioral Relation Learning for Recommendation via Memory Augmented Transformer Network](https://mlsanity.com/paper/2110.04002/) (2021)
- [ClickPrompt: CTR Models are Strong Prompt Generators for Adapting Language Models to CTR Prediction](https://mlsanity.com/paper/2310.09234/) (2023)
- [Transfer-Meta Framework for Cross-domain Recommendation to Cold-Start Users](https://mlsanity.com/paper/2105.04785/) (2021)
- [ARM-Net: Adaptive Relation Modeling Network for Structured Data](https://mlsanity.com/paper/2107.01830/) (2021)
- [CARCA: Context and Attribute-Aware Next-Item Recommendation via Cross-Attention](https://mlsanity.com/paper/2204.06519/) (2022)
- [GREASE: Generate Factual and Counterfactual Explanations for GNN-based Recommendations](https://mlsanity.com/paper/2208.04222/) (2022)
- [Knowledge Graph Self-Supervised Rationalization for Recommendation](https://mlsanity.com/paper/2307.02759/) (2023)
- [Towards Open-World Feature Extrapolation: An Inductive Graph Learning Approach](https://mlsanity.com/paper/2110.04514/) (2021)
- [Efficient Bi-Level Optimization for Recommendation Denoising](https://mlsanity.com/paper/2210.10321/) (2022)
- [Embedding in Recommender Systems: A Survey](https://mlsanity.com/paper/2310.18608/) (2023)
- [DaisyRec 2.0: Benchmarking Recommendation for Rigorous Evaluation](https://mlsanity.com/paper/2206.10848/) (2022)
- [Adap-τ: Adaptively Modulating Embedding Magnitude for Recommendation](https://mlsanity.com/paper/2302.04775/) (2023)
- [STEM: Unleashing the Power of Embeddings for Multi-task Recommendation](https://mlsanity.com/paper/2308.13537/) (2023)
- [ContextGNN: Beyond Two-Tower Recommendation Systems](https://mlsanity.com/paper/2411.19513/) (2024)
- [Concept-Aware Denoising Graph Neural Network for Micro-Video Recommendation](https://mlsanity.com/paper/2109.13527/) (2021)
- [Dynamic Memory based Attention Network for Sequential Recommendation](https://mlsanity.com/paper/2102.09269/) (2021)

## Abstract (arXiv)

Many predictive tasks of web applications need to model categorical variables, such as user IDs and demographics like genders and occupations. To apply standard machine learning techniques, these categorical predictors are always converted to a set of binary features via one-hot encoding, making the resultant feature vector highly sparse. To learn from such sparse data effectively, it is crucial to account for the interactions between features. Factorization Machines (FMs) are a popular solution for efficiently using the second-order feature interactions. However, FM models feature interactions in a linear way, which can be insufficient for capturing the non-linear and complex inherent structure of real-world data. While deep neural networks have recently been applied to learn non-linear feature interactions in industry, such as the Wide&Deep by Google and DeepCross by Microsoft, the deep structure meanwhile makes them difficult to train. In this paper, we propose a novel model Neural Factorization Machine (NFM) for prediction under sparse settings. NFM seamlessly combines the linearity of FM in modelling second-order feature interactions and the non-linearity of neural network in modelling higher-order feature interactions. Conceptually, NFM is more expressive than FM since FM can be seen as a special case of NFM without hidden layers. Empirical results on two regression tasks show that with one hidden layer only, NFM significantly outperforms FM with a 7.3% relative improvement. Compared to the recent deep learning methods Wide&Deep and DeepCross, our NFM uses a shallower structure but offers better performance, being much easier to train and tune in practice.