# Bullion: A Column Store for Machine Learning

arXiv: https://arxiv.org/abs/2404.08901 · HTML: https://mlsanity.com/paper/2404.08901/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 78 references that are in the map.
- First posted on arXiv 13 Apr 2024.

## Builds on (most cited first, 19 of 19)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [In-Datacenter Performance Analysis of a Tensor Processing Unit](https://mlsanity.com/paper/1704.04760/) (2017)
- [Deep Learning Recommendation Model for Personalization and Recommendation Systems](https://mlsanity.com/paper/1906.00091/) (2019)
- [Embedding-based Retrieval in Facebook Search](https://mlsanity.com/paper/2006.11632/) (2020)
- [tf.data: A Machine Learning Data Processing Framework](https://mlsanity.com/paper/2101.12127/) (2021)
- [Understanding Data Storage and Ingestion for Large-Scale Deep Recommendation Model Training](https://mlsanity.com/paper/2108.09373/) (2021)
- [End-to-end Optimization of Machine Learning Prediction Queries](https://mlsanity.com/paper/2206.00136/) (2022)
- [tf.data service: A Case for Disaggregating ML Input Data Processing](https://mlsanity.com/paper/2210.14826/) (2022)
- [Extending Relational Query Processing with ML Inference](https://mlsanity.com/paper/1911.00231/) (2019)
- [Query Processing on Tensor Computation Runtimes](https://mlsanity.com/paper/2203.01877/) (2022)
- [Share the Tensor Tea: How Databases can Leverage the Machine Learning Ecosystem](https://mlsanity.com/paper/2209.04579/) (2022)
- [An Empirical Evaluation of Columnar Storage Formats](https://mlsanity.com/paper/2304.05028/) (2023)
- [The Tensor Data Platform: Towards an AI-centric Database System](https://mlsanity.com/paper/2211.02753/) (2022)
- [Flock: A Low-Cost Streaming Query Engine on FaaS Platforms](https://mlsanity.com/paper/2312.16735/) (2023)
- [SFVInt: Simple, Fast and Generic Variable-Length Integer Decoding using Bit Manipulation Instructions](https://mlsanity.com/paper/2403.06898/) (2024)

## Built on by (most cited first, 1 of 1)

- [Versioned Late Materialization for Ultra-Long Sequence Training in Recommendation Systems at Scale](https://mlsanity.com/paper/2604.24806/) (2026)

## Abstract (arXiv)

The past two decades have witnessed significant success in applying columnar storage to data warehousing and analytics. However, the rapid growth of machine learning poses new challenges. This paper presents Bullion, a columnar storage system tailored for machine learning workloads. Bullion addresses the complexities of data compliance, optimizes the encoding of long sequence sparse features, efficiently manages wide-table projections, introduces feature quantization in storage, enables quality-aware sequential reads for multimodal training data, and provides a comprehensive cascading encoding framework that unifies diverse encoding schemes through modular, composable interfaces. By aligning with the evolving requirements of ML applications, Bullion facilitates the application of columnar storage and processing to modern application scenarios such as those within advertising, recommendation systems, and Generative AI. Preliminary experimental results and theoretical analysis demonstrate Bullion's improved ability to deliver strong performance in the face of the unique demands of machine learning workloads compared to existing columnar storage solutions. Bullion significantly reduces I/O costs for deletion compliance, achieves substantial storage savings with its optimized encoding scheme for sparse features, and improves metadata parsing speed for wide-table projections. These advancements enable Bullion to become an important component in the future of machine learning infrastructure, enabling organizations to efficiently manage and process the massive volumes of data required for training and inference in modern AI applications.