# Industry Insights from Comparing Deep Learning and GBDT Models for E-Commerce Learning-to-Rank

arXiv: https://arxiv.org/abs/2507.20753 · HTML: https://mlsanity.com/paper/2507.20753/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 22 references that are in the map.
- First posted on arXiv 28 Jul 2025.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Introducing LETOR 4.0 Datasets](https://mlsanity.com/paper/1306.2597/) (2013)
- [Applying Deep Learning To Airbnb Search](https://mlsanity.com/paper/1810.09591/) (2018)
- [On Application of Learning to Rank for E-Commerce Search](https://mlsanity.com/paper/1903.04263/) (2019)
- [Context-Aware Learning to Rank with Self-Attention](https://mlsanity.com/paper/2005.10084/) (2020)
- [Widespread Flaws in Offline Evaluation of Recommender Systems](https://mlsanity.com/paper/2307.14951/) (2023)
- [Towards Better Web Search Performance: Pre-training, Fine-tuning and Learning to Rank](https://mlsanity.com/paper/2303.04710/) (2023)
- [A Large Scale Search Dataset for Unbiased Learning to Rank](https://mlsanity.com/paper/2207.03051/) (2022)
- [Unbiased Learning to Rank Meets Reality: Lessons from Baidu's Large-Scale Search Dataset](https://mlsanity.com/paper/2404.02543/) (2024)
- [RankFormer: Listwise Learning-to-Rank Using Listwide Labels](https://mlsanity.com/paper/2306.05808/) (2023)
- [Scaling Session-Based Transformer Recommendations using Optimized Negative Sampling and Loss Functions](https://mlsanity.com/paper/2307.14906/) (2023)
- [Semantic Retrieval at Walmart](https://mlsanity.com/paper/2412.04637/) (2024)

## Abstract (arXiv)

In e-commerce recommender and search systems, tree-based models, such as LambdaMART, have set a strong baseline for Learning-to-Rank (LTR) tasks. Despite their effectiveness and widespread adoption in industry, the debate continues whether deep neural networks (DNNs) can outperform traditional tree-based models in this domain. To contribute to this discussion, we systematically benchmark DNNs against our production-grade LambdaMART model. We evaluate multiple DNN architectures and loss functions on a proprietary dataset from OTTO and validate our findings through an 8-week online A/B test. The results show that a simple DNN architecture outperforms a strong tree-based baseline in terms of total clicks and revenue, while achieving parity in total units sold.