mlsanityData to 7 Oct 2026

DHEN: A Deep and Hierarchical Ensemble Network for Large-Scale Click-Through Rate Prediction

arXiv 2203.11014 ↗First posted 11 Mar 2022cs.IR cs.AI cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

Learning feature interactions is important to the model performance of online advertising services. As a result, extensive efforts have been devoted to designing effective architectures to learn feature interactions. However, we observe that the practical performance of those designs can vary from dataset to dataset, even when the order of interactions claimed to be captured is the same. That indicates different designs may have different advantages and the interactions captured by them have non-overlapping information. Motivated by this observation, we propose DHEN - a deep and hierarchical ensemble architecture that can leverage strengths of heterogeneous interaction modules and learn a hierarchy of the interactions under different orders. To overcome the challenge brought by DHEN's deeper and multi-layer structure in training, we propose a novel co-designed training system that can further improve the training efficiency of DHEN. Experiments of DHEN on large-scale dataset from CTR prediction tasks attained 0.27% improvement on the Normalized Entropy (NE) of prediction and 1.2x better training throughput than state-of-the-art baseline, demonstrating their effectiveness in practice.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 34 of its 51 references that are in the map. First posted on arXiv 11 Mar 2022.

Builds on

34 of 51 references.

References · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2016Layer Normalizationcited by 5,579
  4. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  5. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  6. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,479
  7. 2016Training Deep Nets with Sublinear Memory Costcited by 825
  8. 2016Wide & Deep Learning for Recommender Systemscited by 745
  9. 2017DeepFM: A Factorization-Machine based Neural Network for CTR Predictioncited by 673
  10. 2017Deep Interest Network for Click-Through Rate Predictioncited by 541
  11. 2021MetaFormer Is Actually What You Need for Visioncited by 523
  12. 2017Deep & Cross Network for Ad Click Predictionscited by 392
22 more
  1. 2016RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanismcited by 344
  2. 2019Deep Learning Recommendation Model for Personalization and Recommendation Systemscited by 327
  3. 2020DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systemscited by 326
  4. 2018AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networkscited by 319
  5. 2018xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systemscited by 277
  6. 2021ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learningcited by 253
  7. 2017Neural Factorization Machines for Sparse Predictive Analyticscited by 183
  8. 2018Multimodal Explanations: Justifying Decisions and Pointing to the Evidencecited by 179
  9. 2016Product-based Neural Networks for User Response Predictioncited by 145
  10. 2017Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networkscited by 126
  11. 2019FiBiNET: Combining Feature Importance and Bilinear feature Interaction for Click-Through Rate Predictioncited by 83
  12. 2017Failures of Gradient-Based Deep Learningcited by 79
  13. 2021Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Modelscited by 78
  14. 2018NVIDIA Tensor Core Programmability, Performance & Precisioncited by 77
  15. 2016Deep Learning over Multi-field Categorical Data: A Case Study on User Response Predictioncited by 67
  16. 2020AutoFIS: Automatic Feature Interaction Selection in Factorization Models for Click-Through Rate Predictioncited by 52
  17. 2016Higher-Order Factorization Machinescited by 45
  18. 2021Deep Learning for Click-Through Rate Estimationcited by 33
  19. 2022Click-Through Rate Prediction in Online Advertising: A Literature Reviewcited by 31
  20. 2019Feature Generation by Convolutional Neural Network for Click-Through Rate Predictioncited by 26
  21. 2017Field-aware Factorization Machines in a Real-world Online Advertising Systemcited by 21
  22. 2021AIM: Automatic Interaction Machine for Click-Through Rate Predictioncited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 6; 2024: 1; 2025: 2; 2026: 2.

12 papers.

  1. 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 641
  2. 2023With Shared Microexponents, A Little Shifting Goes a Long Waycited by 47
  3. 2023MAD Max Beyond Single-Node: Enabling Large Machine Learning Model Acceleration on Distributed Systemscited by 14
  4. 2023Causality-based CTR Prediction using Graph Neural Networkscited by 9
  5. 2024Disaggregated Multi-Tower: Topology-aware Modeling Technique for Efficient Large-Scale Recommendationcited by 6
  6. 2022CowClip: Reducing CTR Prediction Model Training Time from 12 hours to 10 minutes on 1 GPUcited by 4
  7. 2023adSformers: Personalization from Short-Term Sequences and Diversity of Representations in Etsy Adscited by 4
  8. 2025KernelEvolve: Scaling Agentic Kernel Coding for Heterogeneous AI Accelerators at Metacited by 3
  9. 2023DistDNAS: Search Efficient Feature Interactions within 2 Hourscited by 2
  10. 2026LoKA: Low-precision Kernel Applications for Recommendation Models At Scalecited by 1
  11. 2025Beyond Self-Consistency: Loss-Balanced Perturbation-Based Regularization Improves Industrial-Scale Ads Rankingcited by 0
  12. 2026Feed m Birds with One Scone: Accelerating Multi-task Gradient Balancing via Bi-level Optimizationcited by 0