# Exploiting Language Model for Efficient Linguistic Steganalysis

arXiv: https://arxiv.org/abs/2107.12168 · HTML: https://mlsanity.com/paper/2107.12168/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 27 references that are in the map.
- First posted on arXiv 26 Jul 2021.

## Builds on (most cited first, 7 of 7)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Semi-supervised Sequence Learning](https://mlsanity.com/paper/1511.01432/) (2015)
- [Neural Linguistic Steganography](https://mlsanity.com/paper/1909.01496/) (2019)
- [Generating Steganographic Text with LSTMs](https://mlsanity.com/paper/1705.10742/) (2017)
- [TS-RNN: Text Steganalysis Based on Recurrent Neural Networks](https://mlsanity.com/paper/1905.13087/) (2019)

## Built on by (most cited first, 6 of 6)

- [Deep Learning for Steganalysis of Diverse Data Types: A review of methods, taxonomy, challenges and future directions](https://mlsanity.com/paper/2308.04522/) (2023)
- [Semantic-Preserving Linguistic Steganography by Pivot Translation and Semantic-Aware Bins Coding](https://mlsanity.com/paper/2203.03795/) (2022)
- [Autoregressive Linguistic Steganography Based on BERT and Consistency Coding](https://mlsanity.com/paper/2203.13972/) (2022)
- [State-of-the-art Advances of Deep-learning Linguistic Steganalysis Research](https://mlsanity.com/paper/2409.01780/) (2024)
- [General Framework for Reversible Data Hiding in Texts Based on Masked Language Modeling](https://mlsanity.com/paper/2206.10112/) (2022)
- [A Comprehensive Survey on Linguistic Steganography: Methods, Countermeasures, Evaluation, and Challenges](https://mlsanity.com/paper/2608.29077/) (2026)

## Abstract (arXiv)

Recent advances in linguistic steganalysis have successively applied CNN, RNN, GNN and other efficient deep models for detecting secret information in generative texts. These methods tend to seek stronger feature extractors to achieve higher steganalysis effects. However, we have found through experiments that there actually exists significant difference between automatically generated stego texts and carrier texts in terms of the conditional probability distribution of individual words. Such kind of difference can be naturally captured by the language model used for generating stego texts. Through further experiments, we conclude that this ability can be transplanted to a text classifier by pre-training and fine-tuning to improve the detection performance. Motivated by this insight, we propose two methods for efficient linguistic steganalysis. One is to pre-train a language model based on RNN, and the other is to pre-train a sequence autoencoder. The results indicate that the two methods have different degrees of performance gain compared to the randomly initialized RNN, and the convergence speed is significantly accelerated. Moreover, our methods achieved the best performance compared to related works, while providing a solution for real-world scenario where there are more cover texts than stego texts.