# GreedyGD: Enhanced Generalized Deduplication for Direct Analytics in IoT

arXiv: https://arxiv.org/abs/2304.07240 · HTML: https://mlsanity.com/paper/2304.07240/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Apr 2023.

## Built on by (most cited first, 3 of 3)

- [RAGE for the Machine: Image Compression with Low-Cost Random Access for Embedded Applications](https://mlsanity.com/paper/2402.05974/) (2024)
- [dreaMLearning: Data Compression Assisted Machine Learning](https://mlsanity.com/paper/2506.22190/) (2025)
- [MUFFLe: Efficient Model Update Compression via Generalized Deduplication for Federated Learning](https://mlsanity.com/paper/2606.14354/) (2026)

## Abstract (arXiv)

Exponential growth in the amount of data generated by the Internet of Things currently pose significant challenges for data communication, storage and analytics and leads to high costs for organisations hoping to leverage their data. Novel techniques are therefore needed to holistically improve the efficiency of data storage and analytics in IoT systems. The emerging compression technique Generalized Deduplication (GD) has been shown to deliver high compression and enable direct compressed data analytics with low storage and memory requirements. In this paper, we propose a new GD-based data compression algorithm called GreedyGD that is designed for analytics. Compared to existing versions of GD, GreedyGD enables more reliable analytics with less data, while running 11.2x faster and delivering even better compression.