# QueryFormer: Winning Solution for KDD Cup 2026 Tencent UniRec Challenge

arXiv: https://arxiv.org/abs/2609.16548 · HTML: https://mlsanity.com/paper/2609.16548/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 29 references that are in the map.
- First posted on arXiv 15 Sep 2026.

## Builds on (most cited first, 22 of 22)

- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Averaging Weights Leads to Wider Optima and Better Generalization](https://mlsanity.com/paper/1803.05407/) (2018)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [DeepFM: A Factorization-Machine based Neural Network for CTR Prediction](https://mlsanity.com/paper/1703.04247/) (2017)
- [Deep Interest Evolution Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1809.03672/) (2018)
- [Deep Learning Recommendation Model for Personalization and Recommendation Systems](https://mlsanity.com/paper/1906.00091/) (2019)
- [DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems](https://mlsanity.com/paper/2008.13535/) (2020)
- [Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations](https://mlsanity.com/paper/2402.17152/) (2024)
- [Search-based User Interest Modeling with Lifelong Sequential Behavior Data for Click-Through Rate Prediction](https://mlsanity.com/paper/2006.05639/) (2020)
- [Ranking Distillation: Learning Compact Ranking Models With High Performance for Recommender System](https://mlsanity.com/paper/1809.07428/) (2018)
- [Fi-GNN: Modeling Feature Interactions via Graph Neural Networks for CTR Prediction](https://mlsanity.com/paper/1910.05552/) (2019)
- [OneTrans: Unified Feature Interaction and Sequence Modeling with One Transformer in Industrial Recommender](https://mlsanity.com/paper/2510.26104/) (2025)
- [Ensemble Knowledge Distillation for CTR Prediction](https://mlsanity.com/paper/2011.04106/) (2020)
- [UniMixer: A Unified Architecture for Scaling Laws in Recommendation Systems](https://mlsanity.com/paper/2604.00590/) (2026)
- [Exploring Scaling Laws of CTR Model for Online Performance Improvement](https://mlsanity.com/paper/2508.15326/) (2025)
- [TokenFormer: Unify the Multi-Field and Sequential Recommendation Worlds](https://mlsanity.com/paper/2604.13737/) (2026)
- [EST: Towards Efficient Scaling Laws in Click-Through Rate Prediction via Unified Modeling](https://mlsanity.com/paper/2602.10811/) (2026)
- [GAP-Net: Calibrating User Intent via Gated Adaptive Progressive Learning for CTR Prediction](https://mlsanity.com/paper/2601.07613/) (2026)
- [Expand More, Shrink Less: Shaping Effective-Rank Dynamics for Dense Scaling in Recommendation](https://mlsanity.com/paper/2605.23191/) (2026)
- [LENS: A Staged Design for Interaction Granularity in Sequential CTR Prediction](https://mlsanity.com/paper/2605.25583/) (2026)
- [Rec-Distill: An Industrial Distillation Pipeline for Large-Scale Recommendation Models](https://mlsanity.com/paper/2605.29755/) (2026)

## Abstract (arXiv)

Post-click conversion rate (pCVR) prediction requires jointly modeling feature interactions and sequential user behaviors. The KDD Cup 2026 Tencent UniRec Challenge calls for a unified architecture addressing both. We observe that existing unified architectures often generate query tokens---the central information hub---with projection-based multi-layer perceptrons (MLPs), without explicit token-to-query attention for refining the query side. We propose QueryFormer, centered on a stackable unified field--sequence block that bridges non-sequential multi-field features and behavioral sequences, and provide a latency-aware scaling study over view width H, model width, depth, data, and compute. The block generates queries through cross-attention and packs sequence queries into shared-parameter attention. QueryFormer secured 1st place in the Industrial Track, achieving an official test area under the ROC curve (AUC) of 0.83254; a modest post-competition scale-up reached 0.832713. Within our grid, H-scaling improves validation AUC from 0.84540 to 0.84615 and beats HyFormer at comparable budgets. Ablation identifies query generation as the largest contributor. Packed shared-parameter cross-attention keeps H=8 inference latency to only 1.89x that of H=1, positioning the bridge as an efficient stackable unified block.