# Explicit Semantic Cross Feature Learning via Pre-trained Graph Neural Networks for CTR Prediction

arXiv: https://arxiv.org/abs/2105.07752 · HTML: https://mlsanity.com/paper/2105.07752/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 17 references that are in the map.
- First posted on arXiv 17 May 2021.

## Builds on (most cited first, 13 of 13)

- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Strategies for Pre-training Graph Neural Networks](https://mlsanity.com/paper/1905.12265/) (2019)
- [Wide & Deep Learning for Recommender Systems](https://mlsanity.com/paper/1606.07792/) (2016)
- [DeepFM: A Factorization-Machine based Neural Network for CTR Prediction](https://mlsanity.com/paper/1703.04247/) (2017)
- [Deep Interest Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1706.06978/) (2017)
- [Deep & Cross Network for Ad Click Predictions](https://mlsanity.com/paper/1708.05123/) (2017)
- [AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks](https://mlsanity.com/paper/1810.11921/) (2018)
- [Billion-scale Commodity Embedding for E-commerce Recommendation in Alibaba](https://mlsanity.com/paper/1803.02349/) (2018)
- [FiBiNET: Combining Feature Importance and Bilinear feature Interaction for Click-Through Rate Prediction](https://mlsanity.com/paper/1905.09433/) (2019)
- [Fi-GNN: Modeling Feature Interactions via Graph Neural Networks for CTR Prediction](https://mlsanity.com/paper/1910.05552/) (2019)
- [Feature Generation by Convolutional Neural Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1904.04447/) (2019)
- [Graph Structural-topic Neural Network](https://mlsanity.com/paper/2006.14278/) (2020)
- [Cross-GCN: Enhancing Graph Convolutional Network with k-Order Feature Interactions](https://mlsanity.com/paper/2003.02587/) (2020)

## Built on by (most cited first, 6 of 6)

- [COMAE: COMprehensive Attribute Exploration for Zero-shot Hashing](https://mlsanity.com/paper/2402.16424/) (2024)
- [Causality-based CTR Prediction using Graph Neural Networks](https://mlsanity.com/paper/2301.12762/) (2023)
- [Neural Graph Matching for Pre-training Graph Neural Networks](https://mlsanity.com/paper/2203.01597/) (2022)
- [From Scaling to Structured Expressivity: Rethinking Transformers for CTR Prediction](https://mlsanity.com/paper/2511.12081/) (2025)
- [Exploiting Social Graph Networks for Emotion Prediction](https://mlsanity.com/paper/2207.05820/) (2022)
- [Directed Acyclic Graph Factorization Machines for CTR Prediction via Knowledge Distillation](https://mlsanity.com/paper/2211.11159/) (2022)

## Abstract (arXiv)

Cross features play an important role in click-through rate (CTR) prediction. Most of the existing methods adopt a DNN-based model to capture the cross features in an implicit manner. These implicit methods may lead to a sub-optimized performance due to the limitation in explicit semantic modeling. Although traditional statistical explicit semantic cross features can address the problem in these implicit methods, it still suffers from some challenges, including lack of generalization and expensive memory cost. Few works focus on tackling these challenges. In this paper, we take the first step in learning the explicit semantic cross features and propose Pre-trained Cross Feature learning Graph Neural Networks (PCF-GNN), a GNN based pre-trained model aiming at generating cross features in an explicit fashion. Extensive experiments are conducted on both public and industrial datasets, where PCF-GNN shows competence in both performance and memory-efficiency in various tasks.