# Test-Time Steering for Lossless Text Compression via Weighted Product of Experts

arXiv: https://arxiv.org/abs/2511.10660 · HTML: https://mlsanity.com/paper/2511.10660/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 43 references that are in the map.
- First posted on arXiv 4 Nov 2025.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions](https://mlsanity.com/paper/2406.15877/) (2024)
- [DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts](https://mlsanity.com/paper/2105.03023/) (2021)
- [Generalized Product of Experts for Automatic and Principled Fusion of Gaussian Process Predictions](https://mlsanity.com/paper/1410.7827/) (2014)
- [Lossless Compression of Deep Neural Networks](https://mlsanity.com/paper/2001.00218/) (2020)
- [DeepZip: Lossless Data Compression using Recurrent Neural Networks](https://mlsanity.com/paper/1811.08162/) (2018)
- [FineZip : Pushing the Limits of Large Language Models for Practical Lossless Text Compression](https://mlsanity.com/paper/2409.17141/) (2024)
- [STEER: Unified Style Transfer with Expert Reinforcement](https://mlsanity.com/paper/2311.07167/) (2023)
- [Compression via Pre-trained Transformers: A Study on Byte-Level Multimodal Data](https://mlsanity.com/paper/2410.05078/) (2024)
- [AlphaZip: Neural Network-Enhanced Lossless Text Compression](https://mlsanity.com/paper/2409.15046/) (2024)
- [Syntactically Informed Text Compression with Recurrent Neural Networks](https://mlsanity.com/paper/1608.02893/) (2016)

## Built on by (most cited first, 3 of 3)

- [TextEconomizer: Enhancing Lossy Text Compression with Denoising Transformers and Entropy Coding](https://mlsanity.com/paper/2606.08184/) (2026)
- [Evidence-Aligned Local Composition of Discrete Experts for Sequence Restoration](https://mlsanity.com/paper/2609.05801/) (2026)
- [HyperZip: Efficient Data Compression through Personalized Diffusion LLMs with Hypernetworks](https://mlsanity.com/paper/2609.36357/) (2026)

## Abstract (arXiv)

Lossless compression techniques are crucial in an era of rapidly growing data. Traditional universal compressors like gzip offer low computational overhead, high speed, and broad applicability across data distributions. However, they often lead to worse compression rates than modern neural compressors, which leverage large-scale training data to model data distributions more effectively. Despite their advantages, neural compressors struggle to generalize to unseen data. To address this limitation, we propose a novel framework that performs Test-Time Steering via a Weighted Product of Experts (wPoE). At inference, our method adaptively combines a universal compression model with a pretrained neural language model, ensuring the compression rate is at least as good as that of the best individual model. Extensive experiments demonstrate that our approach improves the performance of text compression without requiring fine-tuning. Furthermore, it seamlessly integrates with any autoregressive language model, providing a practical solution for enhancing text compression across diverse data distributions.