mlsanityData to 7 Oct 2026

Understanding DNNs in Feature Interaction Models: A Dimensional Collapse Perspective

arXiv 2604.26489 ↗First posted 29 Apr 2026cs.LG cs.IRcited by 0

Not on the map. Why?

Abstract (arXiv)

DNNs have gained widespread adoption in feature interaction recommendation models. However, there has been a longstanding debate on their roles. On one hand, some works claim that DNNs possess the ability to implicitly capture high-order feature interactions. Conversely, recent studies have highlighted the limitations of DNNs in effectively learning dot products, specifically second-order interactions, let alone higher-order interactions. In this paper, we present a novel perspective to understand the effectiveness of DNNs: their impact on the dimensional robustness of the representations. In particular, we conduct extensive experiments involving both parallel DNNs and stacked DNNs. Our evaluation encompasses an overall study of complete DNN on two feature interaction models, alongside a fine-grained ablation analysis of components within DNNs. Experimental results demonstrate that both parallel and stacked DNNs can effectively mitigate the dimensional collapse of embeddings. Furthermore, a gradient-based theoretical analysis, supported by empirical evidence, uncovers the underlying mechanisms of dimensional collapse.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 23 of its 34 references that are in the map. First posted on arXiv 29 Apr 2026.

Builds on

23 of 34 references.

References · 23

  1. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,013
  2. 2016Wide & Deep Learning for Recommender Systemscited by 745
  3. 2021Understanding Dimensional Collapse in Contrastive Self-supervised Learningcited by 374
  4. 2020DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systemscited by 326
  5. 2018AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networkscited by 319
  6. 2020Self-supervised Graph Learning for Recommendationcited by 305
  7. 2018xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systemscited by 277
  8. 2017Neural Factorization Machines for Sparse Predictive Analyticscited by 183
  9. 2021Are Graph Augmentations Necessary? Simple Graph Contrastive Learning for Recommendationcited by 173
  10. 2016Product-based Neural Networks for User Response Predictioncited by 145
  11. 2022RankMe: Assessing the downstream performance of pretrained self-supervised representations by their rankcited by 145
  12. 2020Neural Collaborative Filtering vs. Matrix Factorization Revisitedcited by 123
11 more
  1. 2021Contrastive Learning for Representation Degeneration Problem in Sequential Recommendationcited by 121
  2. 2019FiBiNET: Combining Feature Importance and Bilinear feature Interaction for Click-Through Rate Predictioncited by 83
  3. 2024Wukong: Towards a Scaling Law for Large-Scale Recommendationcited by 72
  4. 2021SimpleX: A Simple and Strong Baseline for Collaborative Filteringcited by 65
  5. 2018Field-weighted Factorization Machines for Click-Through Rate Prediction in Display Advertisingcited by 59
  6. 2022BARS: Towards Open Benchmarking for Recommender Systemscited by 54
  7. 2021FM2: Field-matrixed Factorization Machines for Recommender Systemscited by 31
  8. 2023On the Embedding Collapse when Scaling up Recommendation Modelscited by 29
  9. 2024Ads Recommendation in a Collapsed and Entangled Worldcited by 13
  10. 2023Towards Mitigating Dimensional Collapse of Representations in Collaborative Filteringcited by 10
  11. 2024Preventing Dimensional Collapse in Self-Supervised Learning via Orthogonality Regularizationcited by 9

Built on by

Not cited yet.