# Distantly Supervised Transformers For E-Commerce Product QA

arXiv: https://arxiv.org/abs/2104.02947 · HTML: https://mlsanity.com/paper/2104.02947/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 33 references that are in the map.
- First posted on arXiv 7 Apr 2021.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Reading Wikipedia to Answer Open-Domain Questions](https://mlsanity.com/paper/1704.00051/) (2017)
- [Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation](https://mlsanity.com/paper/2004.09813/) (2020)
- [CamemBERT: a Tasty French Language Model](https://mlsanity.com/paper/1911.03894/) (2019)
- [Learning Fine-grained Image Similarity with Deep Ranking](https://mlsanity.com/paper/1404.4661/) (2014)
- [AmazonQA: A Review-Based Question Answering Task](https://mlsanity.com/paper/1908.04364/) (2019)
- [Addressing Complex and Subjective Product-Related Queries with Customer Reviews](https://mlsanity.com/paper/1512.06863/) (2015)
- [Product-Aware Answer Generation in E-Commerce Question-Answering](https://mlsanity.com/paper/1901.07696/) (2019)
- [Incorporating External Knowledge into Machine Reading for Generative Question Answering](https://mlsanity.com/paper/1909.02745/) (2019)
- [An Updated Duet Model for Passage Re-ranking](https://mlsanity.com/paper/1903.07666/) (2019)
- [Opinion-aware Answer Generation for Review-driven Question Answering in E-Commerce](https://mlsanity.com/paper/2008.11972/) (2020)
- [Review-Driven Answer Generation for Product-Related Questions in E-Commerce](https://mlsanity.com/paper/1905.01994/) (2019)
- [Answer Ranking for Product-Related Questions via Multiple Semantic Relations Modeling](https://mlsanity.com/paper/2006.15599/) (2020)
- [Less is More: Rejecting Unreliable Reviews for Product Question Answering](https://mlsanity.com/paper/2007.04526/) (2020)

## Built on by (most cited first, 3 of 3)

- [Product Question Answering in E-Commerce: A Survey](https://mlsanity.com/paper/2302.08092/) (2023)
- [Answer Generation for Questions With Multiple Information Sources in E-Commerce](https://mlsanity.com/paper/2111.14003/) (2021)
- [MindFlow+: A Self-Evolving Agent for E-Commerce Customer Service](https://mlsanity.com/paper/2507.18884/) (2025)

## Abstract (arXiv)

We propose a practical instant question answering (QA) system on product pages of ecommerce services, where for each user query, relevant community question answer (CQA) pairs are retrieved. User queries and CQA pairs differ significantly in language characteristics making relevance learning difficult. Our proposed transformer-based model learns a robust relevance function by jointly learning unified syntactic and semantic representations without the need for human labeled data. This is achieved by distantly supervising our model by distilling from predictions of a syntactic matching system on user queries and simultaneously training with CQA pairs. Training with CQA pairs helps our model learning semantic QA relevance and distant supervision enables learning of syntactic features as well as the nuances of user querying language. Additionally, our model encodes queries and candidate responses independently allowing offline candidate embedding generation thereby minimizing the need for real-time transformer model execution. Consequently, our framework is able to scale to large e-commerce QA traffic. Extensive evaluation on user queries shows that our framework significantly outperforms both syntactic and semantic baselines in offline as well as large scale online A/B setups of a popular e-commerce service.