# LeCo: Lightweight Compression via Learning Serial Correlations

arXiv: https://arxiv.org/abs/2306.15374 · HTML: https://mlsanity.com/paper/2306.15374/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 112 references that are in the map.
- First posted on arXiv 27 Jun 2023.

## Builds on (most cited first, 14 of 14)

- [The Case for Learned Index Structures](https://mlsanity.com/paper/1712.01208/) (2017)
- [ALEX: An Updatable Adaptive Learned Index](https://mlsanity.com/paper/1905.08898/) (2019)
- [RadixSpline: A Single-Pass Learned Index](https://mlsanity.com/paper/2004.14541/) (2020)
- [FITing-Tree: A Data-aware Index Structure](https://mlsanity.com/paper/1801.10207/) (2018)
- [SOSD: A Benchmark for Learned Indexes](https://mlsanity.com/paper/1911.13014/) (2019)
- [The PGM-index: a multicriteria, compressed and learned approach to data indexing](https://mlsanity.com/paper/1910.06169/) (2019)
- [CARMI: A Cache-Aware Learned Index with a Cost-based Construction Algorithm](https://mlsanity.com/paper/2103.00858/) (2021)
- [Are Updatable Learned Indexes Ready?](https://mlsanity.com/paper/2207.02900/) (2022)
- [An Empirical Evaluation of Columnar Storage Formats](https://mlsanity.com/paper/2304.05028/) (2023)
- [Squish: Near-Optimal Compression for Archival of Relational Datasets](https://mlsanity.com/paper/1602.04256/) (2016)
- [Order-Preserving Key Compression for In-Memory Search Trees](https://mlsanity.com/paper/2003.02391/) (2020)
- [MorphStore: Analytical Query Engine with a Holistic Compression-Enabled Processing Model](https://mlsanity.com/paper/2004.09350/) (2020)
- [Vectorized VByte Decoding](https://mlsanity.com/paper/1503.07387/) (2015)
- [On Optimally Partitioning Variable-Byte Codes](https://mlsanity.com/paper/1804.10949/) (2018)

## Built on by (most cited first, 3 of 3)

- [A Survey of Learned Indexes for the Multi-dimensional Space](https://mlsanity.com/paper/2403.06456/) (2024)
- [Learned Compression of Nonlinear Time Series With Random Access](https://mlsanity.com/paper/2412.16266/) (2024)
- [Learned Static Function Data Structures](https://mlsanity.com/paper/2510.27588/) (2025)

## Abstract (arXiv)

Lightweight data compression is a key technique that allows column stores to exhibit superior performance for analytical queries. Despite a comprehensive study on dictionary-based encodings to approach Shannon's entropy, few prior works have systematically exploited the serial correlation in a column for compression. In this paper, we propose LeCo (i.e., Learned Compression), a framework that uses machine learning to remove the serial redundancy in a value sequence automatically to achieve an outstanding compression ratio and decompression performance simultaneously. LeCo presents a general approach to this end, making existing (ad-hoc) algorithms such as Frame-of-Reference (FOR), Delta Encoding, and Run-Length Encoding (RLE) special cases under our framework. Our microbenchmark with three synthetic and six real-world data sets shows that a prototype of LeCo achieves a Pareto improvement on both compression ratio and random access speed over the existing solutions. When integrating LeCo into widely-used applications, we observe up to 5.2x speed up in a data analytical query in the Arrow columnar execution engine and a 16% increase in RocksDB's throughput.