mlsanityData to 14 Oct 2026

On-Device Commercial Intent Retrieval Under Size, Latency, and Privacy Constraints: A 3 MiB Retrieval System with Typed Egress Boundaries

arXiv 2610.00170 ↗First posted 16 Sep 2026cs.IR cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

We study commercial intent inference that runs entirely on the user's device, under three constraints frozen before the work began: the downloaded payload under 3 MiB, Tier-0 inference under 20 ms at p95, and no raw text, content embedding, or stable identifier leaving the device. Under them we build a retrieval path over a 6,020-leaf commercial taxonomy: a static embedding table distilled from a Korean sentence transformer, quantized to 4 bits, no inference runtime. Our main result is where that constraint costs accuracy. On real Korean commerce text labelled by others (22,900 AI-Hub shopping reviews), mid-category top-5 on real product names is 75.0% against an 18.4% permutation baseline, but splits on one observable: a query containing some leaf name as a substring scores 83.5%, one containing none 45.2%. A generic 196.6x larger teacher seemed to localize the gap (+20.1 pp without an anchor, +0.1 with). That null was two effects cancelling: the same teacher fine-tuned on the student's own contrastive pairs reaches 0.8586 and beats the pure-encoder student by +10.6 pp with an anchor and +20.9 pp without. The cost is not uniform, but it is not free anywhere; where the anchor is absent, task adaptation buys the teacher nothing, so what the constrained encoder lacks there is capacity. The expensive regime is detectable on-device from the ranker's own score margin: declining the least confident fifth lifts the rest to 0.8296. A second axis we first reported, a manufacturer model code, does not survive source-category fixed effects (-4.0 pp, p=0.51); the anchor does (+13.0 pp). Payload is 2,942,652 bytes, all three library links measured. Tier-0 p95 is 4.431 and 3.670 ms on two iPhones (A14, A16) and 5.080 ms on a budget Android tablet (Snapdragon 695), all slower than three server CPUs on the same code. Taxonomy supervision is mostly synthetic Korean utterances.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 49 references that are in the map. First posted on arXiv 16 Sep 2026.

Builds on

25 of 49 references.

References · 25

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  5. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  6. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  7. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  8. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  9. 2019Advances and Open Problems in Federated Learningcited by 3,046
  10. 2016Deep Learning with Differential Privacycited by 2,728
  11. 2017Billion-scale similarity search with GPUscited by 2,497
  12. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
13 more
  1. 2016Enriching Word Vectors with Subword Informationcited by 1,583
  2. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
  3. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  4. 2019Federated Learning with Personalization Layerscited by 541
  5. 2020MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devicescited by 482
  6. 2021ABO: Dataset and Benchmarks for Real-World 3D Object Understandingcited by 366
  7. 2016Concentrated Differential Privacy: Simplifications, Extensions, and Lower Boundscited by 346
  8. 2022Matryoshka Representation Learningcited by 282
  9. 2014RAPPOR: Randomized Aggregatable Privacy-Preserving Ordinal Responsecited by 272
  10. 2021KLUE: Korean Language Understanding Evaluationcited by 114
  11. 2023Federated Learning of Gboard Language Models with Differential Privacycited by 66
  12. 2022Shopping Queries Dataset: A Large-Scale ESCI Benchmark for Improving Product Searchcited by 65
  13. 2026Timing-Aware Repurchase Prediction for Web-Scale E-Commerce: Survival Models for Multi-Surface Grocery Recommendationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond Matching: Category-Guided Latent Intent Reasoning for Generative Retrieval in E-Commerce
  2. 2026Improving Item Discoverability in e-Commerce Search via Related Intent Generation
  3. 2026Synthetic Data Powers Product Retrieval for Long-tail Knowledge-Intensive Queries in E-commerce Search
  4. 2026Dear Algo: A Precision-First Agentic Intent Layer for Unified Search and Recommendation
  5. 2025Test-Time Scaling Strategies for Generative Retrieval in Multimodal Conversational Recommendations
  6. 2026Agentic Multi-Source Grounding for Enhanced Query Intent Understanding: A DoorDash Case Study
  7. 2023I3: Intent-Introspective Retrieval Conditioned on Instructions
  8. 2026When Context Misleads: Intent-Guided Decoding for Robust Retrieval-Augmented Generation