mlsanityData to 14 Oct 2026

From Scaling to Structured Expressivity: Rethinking Transformers for CTR Prediction

arXiv 2511.12081 ↗First posted 15 Nov 2025cs.IR cs.LGcited by 3

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Despite massive investments in scale, deep models for click-through rate (CTR) prediction often exhibit rapidly diminishing returns -- a stark contrast to the predictable scaling laws seen in large language models (LLMs). We identify the root cause as a fundamental textitstructural misalignment: standard Transformers assume sequential compositionality, whereas CTR data demand combinatorial reasoning over heterogeneous fields. To restore alignment, we introduce the Field-Aware Transformer (FAT). By reconstructing the standard Transformer block with field-centric parameters, FAT achieves textitstructured expressivity, fundamentally shifting the model complexity dependence from the total vocabulary size n with the number of fields F (n gg F). Crucially, to decouple model capacity from field cardinality, FAT employs a Basis-Composed Hypernetwork to synthesize field-specific parameters from shared bases, further reducing parameter complexity. Theoretically, we ground this scaling behavior through a formal scaling law based on Rademacher complexity. Empirically, FAT outperforms exisiting state-of-the-art methods with up to textbf+4.38% AUC improvement, and delivers +2.33% CTR and +0.66% RPM in live production. Our work establishes that scalable recommendation arises not from size alone, but from textitstructured expressivity -- architectural coherence with data semantics.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 21 of its 34 references that are in the map. First posted on arXiv 15 Nov 2025.

Builds on

21 of 34 references.

References · 21

  1. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  2. 2020Scaling Laws for Neural Language Modelscited by 6,586
  3. 2018Self-Attentive Sequential Recommendationcited by 1,184
  4. 2019BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformercited by 964
  5. 2017DeepFM: A Factorization-Machine based Neural Network for CTR Predictioncited by 676
  6. 2017Deep Interest Network for Click-Through Rate Predictioncited by 544
  7. 2020DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systemscited by 327
  8. 2018AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networkscited by 320
  9. 2024Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendationscited by 188
  10. 2024Wukong: Towards a Scaling Law for Large-Scale Recommendationcited by 74
  11. 2024HLLM: Enhancing Sequential Recommendations via Hierarchical Large Language Models for Item and User Modelingcited by 63
  12. 2018Field-weighted Factorization Machines for Click-Through Rate Prediction in Display Advertisingcited by 59
9 more
  1. 2025RankMixer: Scaling Up Ranking Models in Industrial Recommenderscited by 42
  2. 2024Analysing The Impact of Sequence Composition on Language Model Pre-Trainingcited by 25
  3. 2024Multi-Behavior Generative Recommendationcited by 15
  4. 2024Inductive Generative Recommendation via Retrieval-based Speculationcited by 15
  5. 2025Unlocking Scaling Law in Industrial Recommendation Systems with a Three-step Paradigm based Large User Modelcited by 15
  6. 2023Hiformer: Heterogeneous Feature Interactions Learning with Transformers for Recommender Systemscited by 14
  7. 2024Scaling New Frontiers: Insights into Large Recommendation Modelscited by 13
  8. 2025Scaling Transformers for Discriminative Recommendation via Generative Pretrainingcited by 10
  9. 2021Explicit Semantic Cross Feature Learning via Pre-trained Graph Neural Networks for CTR Predictioncited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026UniMixer: A Unified Architecture for Scaling Laws in Recommendation Systemscited by 7
  2. 2026MDL: A Unified Multi-Distribution Learner in Large-scale Industrial Recommendation through Tokenizationcited by 0
  3. 2026SpecFormer: Mitigating Embedding and Attention Collapse via Spectral-Aware Transformer for Recommendationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Selective Test-Time Compute Scaling for Click-Through Rate Prediction via Uncertainty-Triggered Feature Path Exploration
  2. 2026LoopCTR: Unlocking the Loop Scaling Power for Click-Through Rate Prediction
  3. 2026Self-Balancing Gradient Allocation for Heterogeneity-Aware Feature Generation in Click-Through Rate Prediction
  4. 2024RAT: Retrieval-Augmented Transformer for Click-Through Rate Prediction
  5. 2023FLIP: Fine-grained Alignment between ID-based Models and Pretrained Language Models for CTR Prediction
  6. 2023MAP: A Model-agnostic Pretraining Framework for Click-through Rate Prediction
  7. 2022An Incremental Learning framework for Large-scale CTR Prediction
  8. 2026Hidden Decoding at Scale: Latent Computation Scaling for Large Language Models