# OD-Stega: LLM-Based Relatively Secure Steganography via Optimized Distributions

arXiv: https://arxiv.org/abs/2410.04328 · HTML: https://mlsanity.com/paper/2410.04328/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 39 references that are in the map.
- First posted on arXiv 6 Oct 2024.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Neural Linguistic Steganography](https://mlsanity.com/paper/1909.01496/) (2019)
- [Generating Steganographic Text with LSTMs](https://mlsanity.com/paper/1705.10742/) (2017)
- [Perfectly Secure Steganography Using Minimum Entropy Coupling](https://mlsanity.com/paper/2210.14889/) (2022)
- [Provably Secure Generative Linguistic Steganography](https://mlsanity.com/paper/2106.02011/) (2021)
- [Towards Near-imperceptible Steganographic Text](https://mlsanity.com/paper/1907.06679/) (2019)
- [Near-imperceptible Neural Linguistic Steganography via Self-Adjusting Arithmetic Coding](https://mlsanity.com/paper/2010.00677/) (2020)
- [Towards Next-Generation Steganalysis: LLMs Unleash the Power of Detecting Steganography](https://mlsanity.com/paper/2405.09090/) (2024)
- [An Approach for Text Steganography Based on Markov Chains](https://mlsanity.com/paper/1409.0915/) (2014)

## Built on by (most cited first, 5 of 5)

- [TrojanStego: Your Language Model Can Secretly Be A Steganographic Privacy Leaking Agent](https://mlsanity.com/paper/2505.20118/) (2025)
- [A High-Capacity and Secure Disambiguation Algorithm for Neural Linguistic Steganography](https://mlsanity.com/paper/2510.02332/) (2025)
- [An LLM Framework For Cryptography Over Chat Channels](https://mlsanity.com/paper/2504.08871/) (2025)
- [Text Steganography with Dynamic Codebook and Multimodal Large Language Model](https://mlsanity.com/paper/2604.20269/) (2026)
- [Now You (Still) See Me: Detecting Evasive Steganographic Payloads in LLMs](https://mlsanity.com/paper/2606.09411/) (2026)

## Abstract (arXiv)

We consider coverless steganography where a Large Language Model (LLM) is used to generate stego-texts in combination with arithmetic coding. An efficient method should embed secret bits in as few language tokens as possible while keeping the stego-text as natural as possible. We show that this problem is equivalent to maximizing the entropy of a replacement probability distribution of the next token generation, subject to a constraint on the divergence between the new distribution and the original one produced by the LLM. A closed-form solution is provided under either the KL divergence or the total variation constraint. Several important practical issues are also tackled: 1) An often-overlooked tokenization mismatch issue is resolved with a simple prompt selection approach, 2) The combination of the optimized distribution and the vocabulary truncation technique is considered, and 3) The incorporation of the proposed approach with existing (potentially non arithmetic coding based) techniques, e.g., the Discop technique.