# SHERLOCK: Towards Dynamic Knowledge Adaptation in LLM-enhanced E-commerce Risk Management

arXiv: https://arxiv.org/abs/2510.08948 · HTML: https://mlsanity.com/paper/2510.08948/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 49 references that are in the map.
- First posted on arXiv 10 Oct 2025.

## Builds on (most cited first, 30 of 32)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [LIMA: Less Is More for Alignment](https://mlsanity.com/paper/2305.11206/) (2023)
- [M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation](https://mlsanity.com/paper/2402.03216/) (2024)
- [Towards Mitigating Hallucination in Large Language Models via Self-Reflection](https://mlsanity.com/paper/2310.06271/) (2023)
- [Injecting New Knowledge into Large Language Models via Supervised Fine-Tuning](https://mlsanity.com/paper/2404.00213/) (2024)
- [Empowering LLMs with Logical Reasoning: A Comprehensive Survey](https://mlsanity.com/paper/2502.15652/) (2025)
- [EcomGPT: Instruction-tuning Large Language Models with Chain-of-Task Tasks for E-commerce](https://mlsanity.com/paper/2308.06966/) (2023)
- [eCeLLM: Generalizing Large Language Models for E-commerce from Large-scale, High-quality Instruction Data](https://mlsanity.com/paper/2402.08831/) (2024)
- [Product Knowledge Graph Embedding for E-commerce](https://mlsanity.com/paper/1911.12481/) (2019)
- [Explainable AI: current status and future directions](https://mlsanity.com/paper/2107.07045/) (2021)
- [Fraud-R1 : A Multi-Round Benchmark for Assessing the Robustness of LLM Against Augmented Fraud and Phishing Inducements](https://mlsanity.com/paper/2502.12904/) (2025)
- [xFraud: Explainable Fraud Transaction Detection](https://mlsanity.com/paper/2011.12193/) (2020)
- [TasTe: Teaching Large Language Models to Translate through Self-Reflection](https://mlsanity.com/paper/2406.08434/) (2024)
- [ChatRule: Mining Logical Rules with Large Language Models for Knowledge Graph Reasoning](https://mlsanity.com/paper/2309.01538/) (2023)
- [Arena Learning: Build Data Flywheel for LLMs Post-training via Simulated Chatbot Arena](https://mlsanity.com/paper/2407.10627/) (2024)
- [Bootstrapping Language-Guided Navigation Learning with Self-Refining Data Flywheel](https://mlsanity.com/paper/2412.08467/) (2024)
- [Is Your LLM-Based Multi-Agent a Reliable Real-World Planner? Exploring Fraud Detection in Travel Planning](https://mlsanity.com/paper/2505.16557/) (2025)
- [DGP: A Dual-Granularity Prompting Framework for Fraud Detection with Graph-Enhanced LLMs](https://mlsanity.com/paper/2507.21653/) (2025)
- [TitAnt: Online Real-time Transaction Fraud Detection in Ant Financial](https://mlsanity.com/paper/1906.07407/) (2019)
- [Can LLMs Find Fraudsters? Multi-level LLM Enhanced Graph Fraud Detection](https://mlsanity.com/paper/2507.11997/) (2025)
- [Exploring Self-supervised Logic-enhanced Training for Large Language Models](https://mlsanity.com/paper/2305.13718/) (2023)
- [Investigating LLM Applications in E-Commerce](https://mlsanity.com/paper/2408.12779/) (2024)
- [General2Specialized LLMs Translation for E-commerce](https://mlsanity.com/paper/2403.03689/) (2024)
- [A Time Attention based Fraud Transaction Detection Framework](https://mlsanity.com/paper/1912.11760/) (2019)
- [From the New World of Word Embeddings: A Comparative Study of Small-World Lexico-Semantic Networks in LLMs](https://mlsanity.com/paper/2502.11380/) (2025)

## Built on by (most cited first, 1 of 1)

- [When Language Models Meet NeuroGraphs: Exploring Enhanced Agentic LLM Framework Towards Brain Network Analysis](https://mlsanity.com/paper/2607.22082/) (2026)

## Abstract (arXiv)

Effective e-commerce risk management requires in-depth case investigations to identify emerging fraud patterns in highly adversarial environments. However, manual investigation typically requires analyzing the associations and couplings among multi-source heterogeneous data, a labor-intensive process that limits efficiency. While Large Language Models (LLMs) show promise in automating these analyses, their deployment is hindered by the complexity of risk scenarios and the sparsity of long-tail domain knowledge. To address these challenges, we propose Sherlock, a framework that integrates structured domain knowledge with LLM-based reasoning through three core modules. First, we construct a domain Knowledge Base (KB) by distilling structured expertise from heterogeneous knowledge sources. Second, we design a two-stage retrieval-augmented generation strategy tailored for case investigation, which combines input contextual augmentation with a Reflect & Refine module to fully leverage the KB for improved analysis quality. Finally, we develop an integrated platform for operations and annotation to drive a self-evolving data flywheel. By combining real-time hotfixes through KB updates with periodic logic alignment via post-training, we facilitate continuous system evolution to counteract adversarial drifts. Online A/B tests at JD dot com demonstrate that Sherlock achieves an 82% Expert Acceptance Rate (EAR) and a 386.7% increase in daily investigation throughput. An additional 90-day evaluation shows that the flywheel successfully recovers from performance decay caused by changing tactics twice, raising the EAR ceiling by around 3.5% through autonomous model updates.