mlsanityData to 14 Oct 2026

Beyond Self-Consistency: Loss-Balanced Perturbation-Based Regularization Improves Industrial-Scale Ads Ranking

arXiv 2502.18478 ↗First posted 5 Feb 2025cs.IR cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Perturbation-based regularization techniques address many challenges in industrial-scale large models, particularly with sparse labels, and emphasize consistency and invariance for perturbation in model predictions. One of the popular regularization techniques has been various forms of self-consistency, which involve making small modifications to input data while preserving contextual information and enforcing similar predictions through auxiliary loss functions. In this work, we explore the first successful application of perturbation-based regularization algorithms in large-scale ads ranking models, and further propose a novel regularization algorithm, namely, Loss-Balanced Small Perturbation Regularization (LSPR) that can be used in potentially any deep learning model. We have successfully demonstrate that both Self-Consistency Regularization approaches (SCR) and LSPR are scalable and can improve ads delivery systems. By conducting industrial-scale experiments, and numerical analysis, we additionally show that our proposed LSPR, performs consistently better compared to SCR, across various groups and signal availability setups. Finally, we report a successful application of the proposed LSPR in a billion-scale industrial ranking system, which to the best of our knowledge, is the first of its kind, and it is specially designed to address the various scalability challenges (e.g, various surfaces, geological locations, clients and so on) as we will mention in this paper.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 29 of its 44 references that are in the map. First posted on arXiv 5 Feb 2025.

Builds on

29 of 44 references.

References · 29

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  5. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
  6. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  7. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,838
  8. 2017DeepFM: A Factorization-Machine based Neural Network for CTR Predictioncited by 676
  9. 2020A Survey on Contrastive Self-supervised Learningcited by 554
  10. 2017Deep & Cross Network for Ad Click Predictionscited by 393
  11. 2019Deep Learning Recommendation Model for Personalization and Recommendation Systemscited by 327
  12. 2017Recent Trends in Deep Learning Based Natural Language Processingcited by 263
17 more
  1. 2022DiffCSE: Difference-based Contrastive Learning for Sentence Embeddingscited by 132
  2. 2022Self-Supervised Learning for Recommender Systems: A Surveycited by 95
  3. 2013Dropout Training as Adaptive Regularizationcited by 93
  4. 2021Consistency Regularization for Variational Auto-Encoderscited by 51
  5. 2022Rethinking the Augmentation Module in Contrastive Learning: Learning Hierarchical Augmentation Invariance with Expanded Viewscited by 37
  6. 2021SelfMatch: Combining Contrastive Self-Supervision and Consistency for Semi-Supervised Learningcited by 35
  7. 2022On the Factory Floor: ML Engineering for Industrial-Scale Ads Recommendation Modelscited by 34
  8. 2021Deep Learning for Click-Through Rate Estimationcited by 33
  9. 2021Noise Stability Regularization for Improving BERT Fine-tuningcited by 29
  10. 2022Explicit Regularization in Overparametrized Models via Noise Injectioncited by 26
  11. 2022ConMatch: Semi-Supervised Learning with Confidence-Guided Consistency Regularizationcited by 25
  12. 2021On the Inherent Regularization Effects of Noise Injection During Trainingcited by 18
  13. 2021Consistency Regularization Can Improve Robustness to Label Noisecited by 18
  14. 2022Hyperspherical Consistency Regularizationcited by 16
  15. 2022CL4CTR: A Contrastive Learning Framework for CTR Predictioncited by 14
  16. 2022DHEN: A Deep and Hierarchical Ensemble Network for Large-Scale Click-Through Rate Predictioncited by 12
  17. 2022Feature Dropout: Revisiting the Role of Augmentations in Contrastive Learningcited by 9

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Taming Sensitive Weights : Noise Perturbation Fine-tuning for Robust LLM Quantization
  2. 2023Annealing Self-Distillation Rectification Improves Adversarial Training
  3. 2023Enhancing Generalization of Universal Adversarial Perturbation through Gradient Aggregation
  4. 2025Quantization through Piecewise-Affine Regularization: Optimization and Statistical Guarantees
  5. 2024MagR: Weight Magnitude Reduction for Enhancing Post-Training Quantization
  6. 2024QT-DoG: Quantization-aware Training for Domain Generalization
  7. 2025Stabilizing Quantization-Aware Training by Implicit-Regularization on Hessian Matrix
  8. 2024Regularization for Adversarial Robust Learning