# Higher-Order Factorization Machines

arXiv: https://arxiv.org/abs/1607.07195 · HTML: https://mlsanity.com/paper/1607.07195/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jul 2016.

## Built on by (most cited first, 30 of 45)

- [Hyperbolic Deep Neural Networks: A Survey](https://mlsanity.com/paper/2101.04562/) (2021)
- [Wukong: Towards a Scaling Law for Large-Scale Recommendation](https://mlsanity.com/paper/2403.02545/) (2024)
- [A Survey on Accuracy-oriented Neural Recommendation: From Collaborative Filtering to Information-rich Recommendation](https://mlsanity.com/paper/2104.13030/) (2021)
- [Tensor Methods in Computer Vision and Deep Learning](https://mlsanity.com/paper/2107.03436/) (2021)
- [Deep Learning for Click-Through Rate Estimation](https://mlsanity.com/paper/2104.10584/) (2021)
- [A Survey on Vertical Federated Learning: From a Layered Perspective](https://mlsanity.com/paper/2304.01829/) (2023)
- [Self-optimizing Feature Generation via Categorical Hashing Representation and Hierarchical Reinforcement Crossing](https://mlsanity.com/paper/2309.04612/) (2023)
- [ARM-Net: Adaptive Relation Modeling Network for Structured Data](https://mlsanity.com/paper/2107.01830/) (2021)
- [User Response Prediction in Online Advertising](https://mlsanity.com/paper/2101.02342/) (2021)
- [Learning Graph Meta Embeddings for Cold-Start Ads in Click-Through Rate Prediction](https://mlsanity.com/paper/2105.08909/) (2021)
- [GraphFM: Graph Factorization Machines for Feature Interaction Modeling](https://mlsanity.com/paper/2105.11866/) (2021)
- [DHEN: A Deep and Hierarchical Ensemble Network for Large-Scale Click-Through Rate Prediction](https://mlsanity.com/paper/2203.11014/) (2022)
- [HyperFormer: Learning Expressive Sparse Feature Representations via Hypergraph Transformer](https://mlsanity.com/paper/2305.17386/) (2023)
- [Higher-order Neural Additive Models: An Interpretable Machine Learning Model with Feature Interactions](https://mlsanity.com/paper/2209.15409/) (2022)
- [XCrossNet: Feature Structure-Oriented Learning for Click-Through Rate Prediction](https://mlsanity.com/paper/2104.10907/) (2021)
- [Quantized Fourier and Polynomial Features for more Expressive Tensor Network Models](https://mlsanity.com/paper/2309.05436/) (2023)
- [DCAP: Deep Cross Attentional Product Network for User Response Prediction](https://mlsanity.com/paper/2105.08649/) (2021)
- [Quaternion Factorization Machines: A Lightweight Solution to Intricate Feature Interaction Modelling](https://mlsanity.com/paper/2104.01716/) (2021)
- [Scalable Higher-Order Tensor Product Spline Models](https://mlsanity.com/paper/2402.01090/) (2024)
- [Field Matters: A Lightweight LLM-enhanced Method for CTR Prediction](https://mlsanity.com/paper/2505.14057/) (2025)
- [MoE-MLoRA for Multi-Domain CTR Prediction: Efficient Adaptation with Expert Specialization](https://mlsanity.com/paper/2506.07563/) (2025)
- [Position: Tensor Networks are a Valuable Asset for Green AI](https://mlsanity.com/paper/2205.12961/) (2022)
- [Factorized Structured Regression for Large-Scale Varying Coefficient Models](https://mlsanity.com/paper/2205.13080/) (2022)
- [Boosting Factorization Machines via Saliency-Guided Mixup](https://mlsanity.com/paper/2206.08661/) (2022)
- [GPRec: Bi-level User Modeling for Deep Recommenders](https://mlsanity.com/paper/2410.20730/) (2024)
- [PolySAE: Modeling Feature Interactions in Sparse Autoencoders via Polynomial Decoding](https://mlsanity.com/paper/2602.01322/) (2026)
- [AEFE: Automatic Embedded Feature Engineering for Categorical Features](https://mlsanity.com/paper/2110.09770/) (2021)
- [xDeepInt: a hybrid architecture for modeling the vector-wise and bit-wise feature interactions](https://mlsanity.com/paper/2301.01089/) (2023)
- [Exploiting Field Dependencies for Learning on Categorical Data](https://mlsanity.com/paper/2307.09321/) (2023)
- [DexDeepFM: Ensemble Diversity Enhanced Extreme Deep Factorization Machine Model](https://mlsanity.com/paper/2104.01924/) (2021)

## Abstract (arXiv)

Factorization machines (FMs) are a supervised learning approach that can use second-order feature combinations even when the data is very high-dimensional. Unfortunately, despite increasing interest in FMs, there exists to date no efficient training algorithm for higher-order FMs (HOFMs). In this paper, we present the first generic yet efficient algorithms for training arbitrary-order HOFMs. We also present new variants of HOFMs with shared parameters, which greatly reduce model size and prediction times while maintaining similar accuracy. We demonstrate the proposed approaches on four different link prediction tasks.