# NeurStore: Efficient In-database Deep Learning Model Management System

arXiv: https://arxiv.org/abs/2509.03228 · HTML: https://mlsanity.com/paper/2509.03228/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 47 references that are in the map.
- First posted on arXiv 3 Sep 2025.

## Builds on (most cited first, 18 of 18)

- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs](https://mlsanity.com/paper/1603.09320/) (2016)
- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [Up or Down? Adaptive Rounding for Post-Training Quantization](https://mlsanity.com/paper/2004.10568/) (2020)
- [Quantizing deep convolutional networks for efficient inference: A whitepaper](https://mlsanity.com/paper/1806.08342/) (2018)
- [Post-Training Quantization for Vision Transformer](https://mlsanity.com/paper/2106.14156/) (2021)
- [GOBO: Quantizing Attention-Based NLP Models for Low Latency and Energy Efficient Inference](https://mlsanity.com/paper/2005.03842/) (2020)
- [Fast Approximate Nearest Neighbor Search With The Navigating Spreading-out Graph](https://mlsanity.com/paper/1707.00143/) (2017)
- [A Comprehensive Survey and Experimental Comparison of Graph-Based Approximate Nearest Neighbor Search](https://mlsanity.com/paper/2101.12631/) (2021)
- [End-to-end Optimization of Machine Learning Prediction Queries](https://mlsanity.com/paper/2206.00136/) (2022)
- [MLCask: Efficient Management of Component Evolution in Collaborative Data Analytics Pipelines](https://mlsanity.com/paper/2010.10246/) (2020)
- [Serving Deep Learning Models with Deduplication from Relational Databases](https://mlsanity.com/paper/2201.10442/) (2022)
- [ModelHub: Towards Unified Data and Lifecycle Management for Deep Learning](https://mlsanity.com/paper/1611.06224/) (2016)
- [A Revisit of Hashing Algorithms for Approximate Nearest Neighbor Search](https://mlsanity.com/paper/1612.07545/) (2016)
- [Powering In-Database Dynamic Model Slicing for Structured Data Analytics](https://mlsanity.com/paper/2405.00568/) (2024)
- [NeurDB: An AI-powered Autonomous Data System](https://mlsanity.com/paper/2405.03924/) (2024)
- [NeurDB: On the Design and Implementation of an AI-powered Autonomous Database](https://mlsanity.com/paper/2408.03013/) (2024)

## Built on by (most cited first, 2 of 2)

- [Poodle: Seamlessly Scaling Down Large Language Models with Just-in-Time Model Replacement](https://mlsanity.com/paper/2512.05525/) (2025)
- [TensorDex: A Compact, Tensor-Centric Storage System for Modern AI Models](https://mlsanity.com/paper/2604.17104/) (2026)

## Abstract (arXiv)

With the prevalence of in-database AI-powered analytics, there is an increasing demand for database systems to efficiently manage the ever-expanding number and size of deep learning models. However, existing database systems typically store entire models as monolithic files or apply compression techniques that overlook the structural characteristics of deep learning models, resulting in suboptimal model storage overhead. This paper presents NeurStore, a novel in-database model management system that enables efficient storage and utilization of deep learning models. First, NeurStore employs a tensor-based model storage engine to enable fine-grained model storage within databases. In particular, we enhance the hierarchical navigable small world (HNSW) graph to index tensors, and only store additional deltas for tensors within a predefined similarity threshold to ensure tensor-level deduplication. Second, we propose a delta quantization algorithm that effectively compresses delta tensors, thus achieving a superior compression ratio with controllable model accuracy loss. Finally, we devise a compression-aware model loading mechanism, which improves model utilization performance by enabling direct computation on compressed tensors. Experimental evaluations demonstrate that NeurStore achieves superior compression ratios and competitive model loading throughput compared to state-of-the-art approaches.