# FiBiNet++: Reducing Model Size by Low Rank Feature Interaction Layer for CTR Prediction

arXiv: https://arxiv.org/abs/2209.05016 · HTML: https://mlsanity.com/paper/2209.05016/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 23 references that are in the map.
- First posted on arXiv 12 Sep 2022.

## Builds on (most cited first, 14 of 14)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Wide & Deep Learning for Recommender Systems](https://mlsanity.com/paper/1606.07792/) (2016)
- [DeepFM: A Factorization-Machine based Neural Network for CTR Prediction](https://mlsanity.com/paper/1703.04247/) (2017)
- [Deep & Cross Network for Ad Click Predictions](https://mlsanity.com/paper/1708.05123/) (2017)
- [DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems](https://mlsanity.com/paper/2008.13535/) (2020)
- [AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks](https://mlsanity.com/paper/1810.11921/) (2018)
- [xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems](https://mlsanity.com/paper/1803.05170/) (2018)
- [Product-based Neural Networks for User Response Prediction](https://mlsanity.com/paper/1611.00144/) (2016)
- [Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks](https://mlsanity.com/paper/1708.04617/) (2017)
- [FiBiNET: Combining Feature Importance and Bilinear feature Interaction for Click-Through Rate Prediction](https://mlsanity.com/paper/1905.09433/) (2019)
- [Operation-aware Neural Networks for User Response Prediction](https://mlsanity.com/paper/1904.12579/) (2019)
- [FAT-DeepFFM: Field Attentive Deep Field-aware Factorization Machine](https://mlsanity.com/paper/1905.06336/) (2019)

## Built on by (most cited first, 8 of 8)

- [MerRec: A Large-scale Multipurpose Mercari Dataset for Consumer-to-Consumer Recommendation Systems](https://mlsanity.com/paper/2402.14230/) (2024)
- [On the Practice of Deep Hierarchical Ensemble Network for Ad Conversion Rate Prediction](https://mlsanity.com/paper/2504.08169/) (2025)
- [The Evolution of Embedding Table Optimization and Multi-Epoch Training in Pinterest Ads Conversion](https://mlsanity.com/paper/2505.05605/) (2025)
- [Multi-Faceted Large Embedding Tables for Pinterest Ads Ranking](https://mlsanity.com/paper/2508.05700/) (2025)
- [Polyhedral Conic Classifier for CTR Prediction](https://mlsanity.com/paper/2406.03892/) (2024)
- [NeSHFS: Neighborhood Search with Heuristic-based Feature Selection for Click-Through Rate Prediction](https://mlsanity.com/paper/2409.08703/) (2024)
- [Addressing Information Loss and Interaction Collapse: A Dual Enhanced Attention Framework for Feature Interaction](https://mlsanity.com/paper/2503.11233/) (2025)
- [SpecFormer: Mitigating Embedding and Attention Collapse via Spectral-Aware Transformer for Recommendation](https://mlsanity.com/paper/2607.24025/) (2026)

## Abstract (arXiv)

Click-Through Rate (CTR) estimation has become one of the most fundamental tasks in many real-world applications and various deep models have been proposed. Some research has proved that FiBiNet is one of the best performance models and outperforms all other models on Avazu dataset. However, the large model size of FiBiNet hinders its wider application. In this paper, we propose a novel FiBiNet++ model to redesign FiBiNet's model structure, which greatly reduces model size while further improves its performance. One of the primary techniques involves our proposed "Low Rank Layer" focused on feature interaction, which serves as a crucial driver of achieving a superior compression ratio for models. Extensive experiments on three public datasets show that FiBiNet++ effectively reduces non-embedding model parameters of FiBiNet by 12x to 16x on three datasets. On the other hand, FiBiNet++ leads to significant performance improvements compared to state-of-the-art CTR methods, including FiBiNet.