# EMOFM: Ensemble MLP mOdel with Feature-based Mixers for Click-Through Rate Prediction

arXiv: https://arxiv.org/abs/2310.04482 · HTML: https://mlsanity.com/paper/2310.04482/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 13 references that are in the map.
- First posted on arXiv 6 Oct 2023.

## Builds on (most cited first, 10 of 10)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [DeepFM: A Factorization-Machine based Neural Network for CTR Prediction](https://mlsanity.com/paper/1703.04247/) (2017)
- [A Survey of Transformers](https://mlsanity.com/paper/2106.04554/) (2021)
- [DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems](https://mlsanity.com/paper/2008.13535/) (2020)
- [Open Benchmarking for Click-Through Rate Prediction](https://mlsanity.com/paper/2009.05794/) (2020)
- [Deep Learning for Click-Through Rate Estimation](https://mlsanity.com/paper/2104.10584/) (2021)
- [Towards Understanding the Overfitting Phenomenon of Deep Click-Through Rate Prediction Models](https://mlsanity.com/paper/2209.06053/) (2022)
- [Click-Through Rate Prediction in Online Advertising: A Literature Review](https://mlsanity.com/paper/2202.10462/) (2022)
- [Disentangled Self-Attentive Neural Networks for Click-Through Rate Prediction](https://mlsanity.com/paper/2101.03654/) (2021)

## Abstract (arXiv)

Track one of CTI competition is on click-through rate (CTR) prediction. The dataset contains millions of records and each field-wise feature in a record consists of hashed integers for privacy. For this task, the keys of network-based methods might be type-wise feature extraction and information fusion across different fields. Multi-layer perceptrons (MLPs) are able to extract field feature, but could not efficiently fuse features. Motivated by the natural fusion characteristic of cross attention and the efficiency of transformer-based structures, we propose simple plug-in mixers for field/type-wise feature fusion, and thus construct an field&type-wise ensemble model, namely EMOFM (Ensemble MLP mOdel with Feature-based Mixers). In the experiments, the proposed model is evaluated on the dataset, the optimization process is visualized and ablation studies are explored. It is shown that EMOFM outperforms compared baselines. In the end, we discuss on future work. WARNING: The comparison might not be fair enough since the proposed method is designed for this data in particular while compared methods are not. For example, EMOFM especially takes different types of interactions into consideration while others do not. Anyway, we do hope that the ideas inside our method could help other developers/learners/researchers/thinkers and so on.