# EnCoD: Distinguishing Compressed and Encrypted File Fragments

arXiv: https://arxiv.org/abs/2010.07754 · HTML: https://mlsanity.com/paper/2010.07754/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Oct 2020.

## Built on by (most cited first, 5 of 5)

- [PassFlow: Guessing Passwords with Generative Flows](https://mlsanity.com/paper/2105.06165/) (2021)
- [Reliable Detection of Compressed and Encrypted Data](https://mlsanity.com/paper/2103.17059/) (2021)
- [MalPhase: Fine-Grained Malware Detection Using Network Flow Data](https://mlsanity.com/paper/2106.00541/) (2021)
- [FedComm: Federated Learning as a Medium for Covert Communication](https://mlsanity.com/paper/2201.08786/) (2022)
- [TATTOOED: A Robust Deep Neural Network Watermarking Scheme based on Spread-Spectrum Channel Coding](https://mlsanity.com/paper/2202.06091/) (2022)

## Abstract (arXiv)

Reliable identification of encrypted file fragments is a requirement for several security applications, including ransomware detection, digital forensics, and traffic analysis. A popular approach consists of estimating high entropy as a proxy for randomness. However, many modern content types (e.g. office documents, media files, etc.) are highly compressed for storage and transmission efficiency. Compression algorithms also output high-entropy data, thus reducing the accuracy of entropy-based encryption detectors. Over the years, a variety of approaches have been proposed to distinguish encrypted file fragments from high-entropy compressed fragments. However, these approaches are typically only evaluated over a few, select data types and fragment sizes, which makes a fair assessment of their practical applicability impossible. This paper aims to close this gap by comparing existing statistical tests on a large, standardized dataset. Our results show that current approaches cannot reliably tell apart encryption and compression, even for large fragment sizes. To address this issue, we design EnCoD, a learning-based classifier which can reliably distinguish compressed and encrypted data, starting with fragments as small as 512 bytes. We evaluate EnCoD against current approaches over a large dataset of different data types, showing that it outperforms current state-of-the-art for most considered fragment sizes and data types.