# Contextual Memory-Enhanced Source Coding for Low-SNR Communications

arXiv: https://arxiv.org/abs/2605.04400 · HTML: https://mlsanity.com/paper/2605.04400/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 30 references that are in the map.
- First posted on arXiv 6 May 2026.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Deep Learning Enabled Semantic Communication Systems](https://mlsanity.com/paper/2006.10685/) (2020)
- [Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications](https://mlsanity.com/paper/2207.09353/) (2022)
- [Locally Typical Sampling](https://mlsanity.com/paper/2202.00666/) (2022)
- [Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models](https://mlsanity.com/paper/2601.07372/) (2026)
- [Hash Embeddings for Efficient Word Representations](https://mlsanity.com/paper/1709.03933/) (2017)
- [Error Correction Code Transformer](https://mlsanity.com/paper/2203.14966/) (2022)
- [Rethinking Modern Communication from Semantic Coding to Semantic Communication](https://mlsanity.com/paper/2110.08496/) (2021)
- [Semantic Communication with Adaptive Universal Transformer](https://mlsanity.com/paper/2108.09119/) (2021)
- [Joint Task and Data Oriented Semantic Communications: A Deep Separate Source-channel Coding Scheme](https://mlsanity.com/paper/2302.13580/) (2023)
- [Semantic Communication with Memory](https://mlsanity.com/paper/2303.12335/) (2023)
- [DeepMA: End-to-end Deep Multiple Access for Wireless Image Transmission in Semantic Communication](https://mlsanity.com/paper/2303.11543/) (2023)
- [Adaptive Bit Rate Control in Semantic Communication with Incremental Knowledge-based HARQ](https://mlsanity.com/paper/2203.06634/) (2022)
- [Separate Source Channel Coding Is Still What You Need: An LLM-based Rethinking](https://mlsanity.com/paper/2501.04285/) (2025)
- [A Sequence Repetition Node-Based Successive Cancellation List Decoder for 5G Polar Codes: Algorithm and Implementation](https://mlsanity.com/paper/2205.08857/) (2022)
- [In-Context Source and Channel Coding](https://mlsanity.com/paper/2601.10267/) (2026)

## Abstract (arXiv)

Separate Source-Channel Coding (SSCC) remains vulnerable in noisy text transmission due to the fragility of autoregressive source decoding, especially when Arithmetic Coding (AC) relies on Large Language Model (LLM)-based probability estimation. This letter proposes a Memory-Augmented Source Coding (MASC) scheme that internalizes contextual patterns into a source model. Specifically, MASC employs a shared Parameterized Contextual Memory (PCM) for multi-order n-gram patterns, and a Mixture-of-Memory-Experts Router (MMER) for sparse, hidden-state-dependent routing over memory experts. This adaptive activation refines source probability estimation, shortens codelength, and mitigates decoding sensitivity to residual channel errors. Experiments over Rayleigh fading and AWGN channels demonstrate its effectiveness.