Measurement-Driven Sub-Network Selection for On-Premise Retrieval-Augmented Factory Agents
On the map: Retrieval-augmented generation → Agentic RAG, agents and memory
Abstract (arXiv)
On-premise assistants can give factory workers conversational access to machine documentation, but models capable of the task rarely fit shop-floor hardware. We show that after structural compression and retrieval-grounded adaptation, model size is no longer a reliable predictor of adapted answer quality: general capability falls almost linearly with parameter count, while judged retrieval-augmented answer quality does not. We therefore treat deployment as a post-adaptation selection problem, committing one sub-network per device on judged answer quality and measured on-device throughput under a configurable general-capability floor and memory budget; rules that optimize size, speed, or quality alone each give up capability or throughput. A weight-shared supernetwork trained with sandwich-style in-place distillation keeps this selection inexpensive. In a manufacturing-manual case study, extraction costs 13.7 percent of the unpruned model's judged quality and retrieval-grounded distillation returns it to within 4.6 percent, recovering two thirds of the loss, and the same assistant runs across three heterogeneous edge tiers at 1.3 to 5 watts standby.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Agentic RAG, agents and memory, one of 15 directions in Retrieval-augmented generation. Builds on 19 of its 22 references that are in the map. First posted on arXiv 2 Sep 2026.
Builds on
19 of 22 references.
References · 19
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
- 2019Once-for-All: Train One Network and Specialize it for Efficient Deploymentcited by 787
- 2023Instruction Tuning with GPT-4cited by 686
- 2023LLM-Pruner: On the Structural Pruning of Large Language Modelscited by 650
- 2023C-Pack: Packed Resources For General Chinese Embeddingscited by 578
- 2023Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruningcited by 415
- 2024RAFT: Adapting Language Model to Domain Specific RAGcited by 262
7 more
- 2019Universally Slimmable Networks and Improved Training Techniquescited by 221
- 2020BigNAS: Scaling Up Neural Architecture Search with Big Single-Stage Modelscited by 188
- 2024Compact Language Models via Pruning and Knowledge Distillationcited by 148
- 2024MELTing point: Mobile Evaluation of Language Transformerscited by 36
- 2024HW-GPT-Bench: Hardware-Aware Architecture Benchmark for Language Modelscited by 10
- 2025Agentic AI for Intent-Based Industrial Automationcited by 7
- 2025Where to Begin: Efficient Pretraining via Subnetwork Selection and Distillationcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026CAFE: A Compound-AI Factorial Evaluation Framework
- 2026Don't Offer What Can't Be Done: Deterministic Executability Gating for LLM Skill Selection at Scale
- 2026GRADRAG: Cross-Component Prompt Adaptation for Coordinated Multi-Agent RAG
- 2025Beyond Component Strength: Synergistic Integration and Adaptive Calibration in Multi-Agent RAG Systems
- 2026DAREBench: Deployment-Aware and Reliable Evaluation of Models as Agents
- 2026Corpus2Skill: Distilling Enterprise Knowledge into Navigable Agent Skills for QA and RAG
- 2025AgentChangeBench: A Multi-Dimensional Evaluation Framework for Goal-Shift Robustness in Conversational AI
- 2026From Monolithic Blending to Agentic Orchestration: Dynamic Response for Conversational Assistants at Scale