# InterFormer: Effective Heterogeneous Interaction Learning for Click-Through Rate Prediction

arXiv: https://arxiv.org/abs/2411.09852 · HTML: https://mlsanity.com/paper/2411.09852/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 87 references that are in the map.
- First posted on arXiv 15 Nov 2024.

## Builds on (most cited first, 30 of 41)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel](https://mlsanity.com/paper/2304.11277/) (2023)
- [Deep Interest Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1706.06978/) (2017)
- [Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering](https://mlsanity.com/paper/1602.01585/) (2016)
- [Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding](https://mlsanity.com/paper/1809.07426/) (2018)
- [Deep & Cross Network for Ad Click Predictions](https://mlsanity.com/paper/1708.05123/) (2017)
- [Deep Interest Evolution Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1809.03672/) (2018)
- [DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems](https://mlsanity.com/paper/2008.13535/) (2020)
- [AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks](https://mlsanity.com/paper/1810.11921/) (2018)
- [xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems](https://mlsanity.com/paper/1803.05170/) (2018)
- [DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMs](https://mlsanity.com/paper/2406.01721/) (2024)
- [Fusing Similarity Models with Markov Chains for Sparse Sequential Recommendation](https://mlsanity.com/paper/1609.09152/) (2016)
- [Deep Learning based Recommender System: A Survey and New Perspectives](https://mlsanity.com/paper/1707.07435/) (2017)
- [Behavior Sequence Transformer for E-commerce Recommendation in Alibaba](https://mlsanity.com/paper/1905.06874/) (2019)
- [Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks](https://mlsanity.com/paper/1708.04617/) (2017)
- [Sequential Recommender Systems: Challenges, Progress and Prospects](https://mlsanity.com/paper/2001.04830/) (2019)
- [Discrete-state Continuous-time Diffusion for Graph Generation](https://mlsanity.com/paper/2405.11416/) (2024)
- [Wukong: Towards a Scaling Law for Large-Scale Recommendation](https://mlsanity.com/paper/2403.02545/) (2024)
- [Sequence-Aware Recommender Systems](https://mlsanity.com/paper/1802.08452/) (2018)
- [BARS: Towards Open Benchmarking for Recommender Systems](https://mlsanity.com/paper/2205.09626/) (2022)
- [MaskNet: Introducing Feature-Wise Multiplication to CTR Ranking Models by Instance-Guided Mask](https://mlsanity.com/paper/2102.07619/) (2021)
- [TransAct: Transformer-based Realtime User Action Model for Recommendation at Pinterest](https://mlsanity.com/paper/2306.00248/) (2023)
- [Networked Time Series Imputation via Position-aware Graph Enhanced Variational Autoencoders](https://mlsanity.com/paper/2305.18612/) (2023)
- [FM2: Field-matrixed Factorization Machines for Recommender Systems](https://mlsanity.com/paper/2102.12994/) (2021)
- [Class-Imbalanced Graph Learning without Class Rebalancing](https://mlsanity.com/paper/2308.14181/) (2023)
- [AceKG: A Large-scale Knowledge Graph for Academic Data Mining](https://mlsanity.com/paper/1807.08484/) (2018)

## Built on by (most cited first, 8 of 8)

- [PLANETALIGN: A Comprehensive Python Library for Benchmarking Network Alignment](https://mlsanity.com/paper/2505.21366/) (2025)
- [Kunlun: Establishing Scaling Laws for Massive-Scale Recommendation Systems through Unified Architecture Design](https://mlsanity.com/paper/2602.10016/) (2026)
- [LLaTTE: Scaling Laws for Multi-Stage Sequence Modeling in Large-Scale Ads Recommendation](https://mlsanity.com/paper/2601.20083/) (2026)
- [SOLARIS: Speculative Offloading of Latent-bAsed Representation for Inference Scaling](https://mlsanity.com/paper/2604.12110/) (2026)
- [Graph homophily booster: Reimagining the role of discrete features in heterophilic graph learning](https://mlsanity.com/paper/2602.07256/) (2026)
- [AvAtar: Learning to Align via Active Optimal Transport](https://mlsanity.com/paper/2605.24395/) (2026)
- [Design Once, Deploy at Scale: Template-Driven ML Development for Large Model Ecosystems](https://mlsanity.com/paper/2603.24963/) (2026)
- [CMSL: Constructive Multi-Sequence Learning for Recommendation Systems](https://mlsanity.com/paper/2606.28533/) (2026)

## Abstract (arXiv)

Click-through rate (CTR) prediction, which predicts the probability of a user clicking an ad, is a fundamental task in recommender systems. The emergence of heterogeneous information, such as user profile and behavior sequences, depicts user interests from different aspects. A mutually beneficial integration of heterogeneous information is the cornerstone towards the success of CTR prediction. However, most of the existing methods suffer from two fundamental limitations, including (1) insufficient inter-mode interaction due to the unidirectional information flow between modes, and (2) aggressive information aggregation caused by early summarization, resulting in excessive information loss. To address the above limitations, we propose a novel module named InterFormer to learn heterogeneous information interaction in an interleaving style. To achieve better interaction learning, InterFormer enables bidirectional information flow for mutually beneficial learning across different modes. To avoid aggressive information aggregation, we retain complete information in each data mode and use a separate bridging arch for effective information selection and summarization. Our proposed InterFormer achieves state-of-the-art performance on three public datasets and a large-scale industrial dataset.