# Differentiable Retrieval Augmentation via Generative Language Modeling for E-commerce Query Intent Classification

arXiv: https://arxiv.org/abs/2308.09308 · HTML: https://mlsanity.com/paper/2308.09308/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 38 references that are in the map.
- First posted on arXiv 18 Aug 2023.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [Latent Retrieval for Weakly Supervised Open Domain Question Answering](https://mlsanity.com/paper/1906.00300/) (2019)
- [Generating Sequences With Recurrent Neural Networks](https://mlsanity.com/paper/1308.0850/) (2013)
- [How to Fine-Tune BERT for Text Classification?](https://mlsanity.com/paper/1905.05583/) (2019)
- [Beam Search Strategies for Neural Machine Translation](https://mlsanity.com/paper/1702.01806/) (2017)
- [Regularizing and Optimizing LSTM Language Models](https://mlsanity.com/paper/1708.02182/) (2017)
- [BERT for Joint Intent Classification and Slot Filling](https://mlsanity.com/paper/1902.10909/) (2019)
- [A Survey on Retrieval-Augmented Text Generation](https://mlsanity.com/paper/2202.01110/) (2022)
- [End-to-End Open-Domain Question Answering with BERTserini](https://mlsanity.com/paper/1902.01718/) (2019)
- [Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answering](https://mlsanity.com/paper/1908.08167/) (2019)
- [M6: A Chinese Multimodal Pretrainer](https://mlsanity.com/paper/2103.00823/) (2021)
- [Retrieval-Enhanced Contrastive Vision-Text Models](https://mlsanity.com/paper/2306.07196/) (2023)
- [VGCN-BERT: Augmenting BERT with Graph Embedding for Text Classification](https://mlsanity.com/paper/2004.05707/) (2020)
- [Towards Personalized and Semantic Retrieval: An End-to-End Solution for E-commerce Search via Embedding Learning](https://mlsanity.com/paper/2006.02282/) (2020)
- [Joint Learning of Deep Retrieval Model and Product Quantization based Embedding Index](https://mlsanity.com/paper/2105.03933/) (2021)
- [Query Rewriting via Cycle-Consistent Translation for E-Commerce Search](https://mlsanity.com/paper/2103.00800/) (2021)
- [SLIM: Explicit Slot-Intent Mapping with BERT for Joint Multi-Intent Detection and Slot Filling](https://mlsanity.com/paper/2108.11711/) (2021)
- [Pre-training Tasks for User Intent Detection and Embedding Retrieval in E-commerce Search](https://mlsanity.com/paper/2208.06150/) (2022)

## Built on by (most cited first, 3 of 3)

- [pEBR: A Probabilistic Approach to Embedding Based Retrieval](https://mlsanity.com/paper/2410.19349/) (2024)
- [The Virtuous Cycle: AI-Powered Vector Search and Vector Search-Augmented AI](https://mlsanity.com/paper/2603.09347/) (2026)
- [Improving Item Discoverability in e-Commerce Search via Related Intent Generation](https://mlsanity.com/paper/2607.27172/) (2026)

## Abstract (arXiv)

Retrieval augmentation, which enhances downstream models by a knowledge retriever and an external corpus instead of by merely increasing the number of model parameters, has been successfully applied to many natural language processing (NLP) tasks such as text classification, question answering and so on. However, existing methods that separately or asynchronously train the retriever and downstream model mainly due to the non-differentiability between the two parts, usually lead to degraded performance compared to end-to-end joint training. In this paper, we propose Differentiable Retrieval Augmentation via Generative lANguage modeling(Dragan), to address this problem by a novel differentiable reformulation. We demonstrate the effectiveness of our proposed method on a challenging NLP task in e-commerce search, namely query intent classification. Both the experimental results and ablation study show that the proposed method significantly and reasonably improves the state-of-the-art baselines on both offline evaluation and online A/B test.