# End-to-End Speech to Intent Prediction to improve E-commerce Customer Support Voicebot in Hindi and English

arXiv: https://arxiv.org/abs/2211.07710 · HTML: https://mlsanity.com/paper/2211.07710/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 31 references that are in the map.
- First posted on arXiv 26 Oct 2022.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [Cyclical Learning Rates for Training Neural Networks](https://mlsanity.com/paper/1506.01186/) (2015)
- [Speech Model Pre-training for End-to-End Spoken Language Understanding](https://mlsanity.com/paper/1904.03670/) (2019)
- [Towards end-to-end spoken language understanding](https://mlsanity.com/paper/1802.08395/) (2018)
- [From Audio to Semantics: Approaches to end-to-end spoken language understanding](https://mlsanity.com/paper/1809.09190/) (2018)
- [Joint Contextual Modeling for ASR Correction and Language Understanding](https://mlsanity.com/paper/2002.00750/) (2020)
- [Speech-language Pre-training for End-to-end Spoken Language Understanding](https://mlsanity.com/paper/2102.06283/) (2021)
- [End-to-End Spoken Language Understanding Without Full Transcripts](https://mlsanity.com/paper/2009.14386/) (2020)
- [Improving Spoken Language Understanding By Exploiting ASR N-best Hypotheses](https://mlsanity.com/paper/2001.05284/) (2020)
- [Adapting Pretrained Transformer to Lattices for Spoken Language Understanding](https://mlsanity.com/paper/2011.00780/) (2020)
- [Spoken Language Intent Detection using Confusion2Vec](https://mlsanity.com/paper/1904.03576/) (2019)
- [End-to-end spoken language understanding using transformer networks and self-supervised pre-trained features](https://mlsanity.com/paper/2011.08238/) (2020)
- [Learning Spoken Language Representations with Neural Lattice Language Modeling](https://mlsanity.com/paper/2007.02629/) (2020)

## Built on by (most cited first, 2 of 2)

- [IntentionQA: A Benchmark for Evaluating Purchase Intention Comprehension Abilities of Language Models in E-commerce](https://mlsanity.com/paper/2406.10173/) (2024)
- [Building Accurate Low Latency ASR for Streaming Voice Search](https://mlsanity.com/paper/2305.18596/) (2023)

## Abstract (arXiv)

Automation of on-call customer support relies heavily on accurate and efficient speech-to-intent (S2I) systems. Building such systems using multi-component pipelines can pose various challenges because they require large annotated datasets, have higher latency, and have complex deployment. These pipelines are also prone to compounding errors. To overcome these challenges, we discuss an end-to-end (E2E) S2I model for customer support voicebot task in a bilingual setting. We show how we can solve E2E intent classification by leveraging a pre-trained automatic speech recognition (ASR) model with slight modification and fine-tuning on small annotated datasets. Experimental results show that our best E2E model outperforms a conventional pipeline by a relative ~27% on the F1 score.