# In-Context Source and Channel Coding

arXiv: https://arxiv.org/abs/2601.10267 · HTML: https://mlsanity.com/paper/2601.10267/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 80 references that are in the map.
- First posted on arXiv 15 Jan 2026.

## Builds on (most cited first, 30 of 41)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Deep Learning Enabled Semantic Communication Systems](https://mlsanity.com/paper/2006.10685/) (2020)
- [Language Modeling Is Compression](https://mlsanity.com/paper/2309.10668/) (2023)
- [Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications](https://mlsanity.com/paper/2207.09353/) (2022)
- [Locally Typical Sampling](https://mlsanity.com/paper/2202.00666/) (2022)
- [Reasoning with Sampling: Your Base Model is Smarter Than You Think](https://mlsanity.com/paper/2510.14901/) (2025)
- [Error Correction Code Transformer](https://mlsanity.com/paper/2203.14966/) (2022)
- [LLMZip: Lossless Text Compression using Large Language Models](https://mlsanity.com/paper/2306.04050/) (2023)
- [DeepZip: Lossless Data Compression using Recurrent Neural Networks](https://mlsanity.com/paper/1811.08162/) (2018)
- [A Fast Transformer-based General-Purpose Lossless Compressor](https://mlsanity.com/paper/2203.16114/) (2022)
- [Overview of AI and Communication for 6G Network: Fundamentals, Challenges, and Future Research Opportunities](https://mlsanity.com/paper/2412.14538/) (2024)
- [Rethinking Modern Communication from Semantic Coding to Semantic Communication](https://mlsanity.com/paper/2110.08496/) (2021)
- [Semantic Communication with Adaptive Universal Transformer](https://mlsanity.com/paper/2108.09119/) (2021)
- [A Survey on Semantic Communication Networks: Architecture, Security, and Privacy](https://mlsanity.com/paper/2405.01221/) (2024)
- [Joint Task and Data Oriented Semantic Communications: A Deep Separate Source-channel Coding Scheme](https://mlsanity.com/paper/2302.13580/) (2023)
- [An Iterative BP-CNN Architecture for Channel Decoding](https://mlsanity.com/paper/1707.05697/) (2017)
- [Semantic Communications for Image Recovery and Classification via Deep Joint Source and Channel Coding](https://mlsanity.com/paper/2304.02317/) (2023)
- [FineZip : Pushing the Limits of Large Language Models for Practical Lossless Text Compression](https://mlsanity.com/paper/2409.17141/) (2024)
- [Time series compression: a survey](https://mlsanity.com/paper/2101.08784/) (2021)
- [QUEST: Quality-Aware Metropolis-Hastings Sampling for Machine Translation](https://mlsanity.com/paper/2406.00049/) (2024)
- [Autoregressive Belief Propagation for Decoding Block Codes](https://mlsanity.com/paper/2103.11780/) (2021)
- [Semantic Communication with Memory](https://mlsanity.com/paper/2303.12335/) (2023)
- [Perceptual Learned Source-Channel Coding for High-Fidelity Image Semantic Transmission](https://mlsanity.com/paper/2205.13120/) (2022)
- [DeepMA: End-to-end Deep Multiple Access for Wireless Image Transmission in Semantic Communication](https://mlsanity.com/paper/2303.11543/) (2023)
- [Adaptive Bit Rate Control in Semantic Communication with Incremental Knowledge-based HARQ](https://mlsanity.com/paper/2203.06634/) (2022)
- [Boosting Learning for LDPC Codes to Improve the Error-Floor Performance](https://mlsanity.com/paper/2310.07194/) (2023)
- [Process-and-Forward: Deep Joint Source-Channel Coding Over Cooperative Relay Networks](https://mlsanity.com/paper/2403.10613/) (2024)
- [Active Deep Decoding of Linear Codes](https://mlsanity.com/paper/1906.02778/) (2019)

## Built on by (most cited first, 1 of 1)

- [Contextual Memory-Enhanced Source Coding for Low-SNR Communications](https://mlsanity.com/paper/2605.04400/) (2026)

## Abstract (arXiv)

Separate Source-Channel Coding (SSCC) remains attractive for text transmission due to its modularity and compatibility with mature entropy coders and powerful channel codes. However, SSCC often suffers from a pronounced cliff effect in low Signal-to-Noise Ratio (SNR) regimes, where residual bit errors after channel decoding can catastrophically break lossless source decoding, especially for Arithmetic Coding (AC) driven by Large Language Models (LLMs). This paper proposes a receiver-side In-Context Decoding (ICD) framework that enhances SSCC robustness without modifying the transmitter. ICD leverages an Error Correction Code Transformer (ECCT) to obtain bit-wise reliability for the decoded information bits. Based on the context-consistent bitstream, ICD constructs a confidence-ranked candidate pool via reliability-guided bit flipping, samples a compact yet diverse subset of candidates, and applies an LLM-based arithmetic decoder to obtain both reconstructions and sequence-level log-likelihoods. A reliability-likelihood fusion rule then selects the final output. We further provide theoretical guarantees on the stability and convergence of the proposed sampling procedure. Extensive experiments over Additive White Gaussian Noise (AWGN) and Rayleigh fading channels demonstrate consistent gains compared with conventional SSCC baselines and representative Joint Source-Channel Coding (JSCC) schemes.