# Investigating LLM Applications in E-Commerce

arXiv: https://arxiv.org/abs/2408.12779 · HTML: https://mlsanity.com/paper/2408.12779/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 32 references that are in the map.
- First posted on arXiv 23 Aug 2024.

## Builds on (most cited first, 14 of 14)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [Aligning AI With Shared Human Values](https://mlsanity.com/paper/2008.02275/) (2020)
- [A Survey on In-context Learning](https://mlsanity.com/paper/2301.00234/) (2022)
- [The Falcon Series of Open Language Models](https://mlsanity.com/paper/2311.16867/) (2023)
- [Shopping Queries Dataset: A Large-Scale ESCI Benchmark for Improving Product Search](https://mlsanity.com/paper/2206.06588/) (2022)
- [Legal Prompt Engineering for Multilingual Legal Judgement Prediction](https://mlsanity.com/paper/2212.02199/) (2022)
- [On the Educational Impact of ChatGPT: Is Artificial Intelligence Ready to Obtain a University Degree?](https://mlsanity.com/paper/2303.11146/) (2023)
- [SeqScore: Addressing Barriers to Reproducible Named Entity Recognition Evaluation](https://mlsanity.com/paper/2107.14154/) (2021)
- [QueryNER: Segmentation of E-commerce Queries](https://mlsanity.com/paper/2405.09507/) (2024)

## Built on by (most cited first, 7 of 7)

- [Fraud-R1 : A Multi-Round Benchmark for Assessing the Robustness of LLM Against Augmented Fraud and Phishing Inducements](https://mlsanity.com/paper/2502.12904/) (2025)
- [Unsafe LLM-Based Search: Quantitative Analysis and Mitigation of Safety Risks in AI Web Search](https://mlsanity.com/paper/2502.04951/) (2025)
- [EVADE-Bench: Multimodal Benchmark for Evaluating and Enhancing Evasive Content Detection](https://mlsanity.com/paper/2505.17654/) (2025)
- [Mix-Ecom: Towards Mixed-Type E-Commerce Dialogues with Complex Domain Rules](https://mlsanity.com/paper/2509.23836/) (2025)
- [Vocabulary Customization for Efficient Domain-Specific LLM Deployment](https://mlsanity.com/paper/2509.26124/) (2025)
- [SHERLOCK: Towards Dynamic Knowledge Adaptation in LLM-enhanced E-commerce Risk Management](https://mlsanity.com/paper/2510.08948/) (2025)
- [Structuring Reasoning for Complex Rules Beyond Flat Representations](https://mlsanity.com/paper/2510.05134/) (2025)

## Abstract (arXiv)

The emergence of Large Language Models (LLMs) has revolutionized natural language processing in various applications especially in e-commerce. One crucial step before the application of such LLMs in these fields is to understand and compare the performance in different use cases in such tasks. This paper explored the efficacy of LLMs in the e-commerce domain, focusing on instruction-tuning an open source LLM model with public e-commerce datasets of varying sizes and comparing the performance with the conventional models prevalent in industrial applications. We conducted a comprehensive comparison between LLMs and traditional pre-trained language models across specific tasks intrinsic to the e-commerce domain, namely classification, generation, summarization, and named entity recognition (NER). Furthermore, we examined the effectiveness of the current niche industrial application of very large LLM, using in-context learning, in e-commerce specific tasks. Our findings indicate that few-shot inference with very large LLMs often does not outperform fine-tuning smaller pre-trained models, underscoring the importance of task-specific model optimization.Additionally, we investigated different training methodologies such as single-task training, mixed-task training, and LoRA merging both within domain/tasks and between different tasks. Through rigorous experimentation and analysis, this paper offers valuable insights into the potential effectiveness of LLMs to advance natural language processing capabilities within the e-commerce industry.