mlsanityData to 14 Oct 2026

SML:Enhance the Network Smoothness with Skip Meta Logit for CTR Prediction

arXiv 2210.10725 ↗First posted 9 Oct 2022cs.IR cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

In light of the smoothness property brought by skip connections in ResNet, this paper proposed the Skip Logit to introduce the skip connection mechanism that fits arbitrary DNN dimensions and embraces similar properties to ResNet. Meta Tanh Normalization (MTN) is designed to learn variance information and stabilize the training process. With these delicate designs, our Skip Meta Logit (SML) brought incremental boosts to the performance of extensive SOTA ctr prediction models on two real-world datasets. In the meantime, we prove that the optimization landscape of arbitrarily deep skip logit networks has no spurious local optima. Finally, SML can be easily added to building blocks and has delivered offline accuracy and online business metrics gains on app ads learning to rank systems at TikTok.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 34 of its 45 references that are in the map. First posted on arXiv 9 Oct 2022.

Builds on

34 of 45 references.

References · 34

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  5. 2016Layer Normalizationcited by 5,669
  6. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
  7. 2016Identity Mappings in Deep Residual Networkscited by 1,965
  8. 2018Group Normalizationcited by 1,524
  9. 2016Instance Normalization: The Missing Ingredient for Fast Stylizationcited by 1,198
  10. 2017Visualizing the Loss Landscape of Neural Netscited by 1,130
  11. 2014Deep Learning in Neural Networks: An Overviewcited by 770
  12. 2016Wide & Deep Learning for Recommender Systemscited by 747
22 more
  1. 2017DeepFM: A Factorization-Machine based Neural Network for CTR Predictioncited by 676
  2. 2015Empirical Evaluation of Rectified Activations in Convolutional Networkcited by 519
  3. 2021Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depthcited by 419
  4. 2017Deep & Cross Network for Ad Click Predictionscited by 393
  5. 2020DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systemscited by 327
  6. 2018AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networkscited by 320
  7. 2016On the Expressive Power of Deep Neural Networkscited by 306
  8. 2018xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systemscited by 278
  9. 2019Fixup Initialization: Residual Learning Without Normalizationcited by 198
  10. 2013Maxout Networkscited by 187
  11. 2017Neural Factorization Machines for Sparse Predictive Analyticscited by 183
  12. 2017The Shattered Gradients Problem: If resnets are the answer, then what is the question?cited by 157
  13. 2016Product-based Neural Networks for User Response Predictioncited by 145
  14. 2017Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networkscited by 126
  15. 2016Identity Matters in Deep Learningcited by 109
  16. 2017Online Deep Learning: Learning Deep Neural Networks on the Flycited by 86
  17. 2021Adversarially Adaptive Normalization for Single Domain Generalizationcited by 85
  18. 2017Failures of Gradient-Based Deep Learningcited by 80
  19. 2019Why ResNet Works? Residuals Generalizecited by 39
  20. 2019Online Normalization for Training Neural Networkscited by 29
  21. 2019Towards Understanding the Importance of Shortcut Connections in Residual Networkscited by 18
  22. 2019Instance-Level Meta Normalizationcited by 7

Built on by

Not cited yet.