# Interplay Between Belief Propagation and Transformer: Differential-Attention Message Passing Transformer

arXiv: https://arxiv.org/abs/2509.15637 · HTML: https://mlsanity.com/paper/2509.15637/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Sep 2025.

## Built on by (most cited first, 6 of 6)

- [Generalization Bounds for Transformer Channel Decoders](https://mlsanity.com/paper/2601.06969/) (2026)
- [Syndrome-Flow Consistency Model Achieves One-step Denoising Error Correction Codes](https://mlsanity.com/paper/2512.01389/) (2025)
- [On the Fragility of AI-Based Channel Decoders under Small Channel Perturbations](https://mlsanity.com/paper/2602.01582/) (2026)
- [Leveraging Code Automorphisms for Improved Syndrome-Based Neural Decoding](https://mlsanity.com/paper/2605.03620/) (2026)
- [Score Based Error Correcting Code Decoder](https://mlsanity.com/paper/2605.28358/) (2026)
- [Neural expectation-maximization and meta-models](https://mlsanity.com/paper/2610.12741/) (2026)

## Abstract (arXiv)

Transformer-based neural decoders have emerged as a promising approach to error correction coding, combining data-driven adaptability with efficient modeling of long-range dependencies. This paper presents a novel decoder architecture that integrates classical belief propagation principles with transformer designs. We introduce a differentiable syndrome loss function leveraging global codebook structure and a differential-attention mechanism optimizing bit and syndrome embedding interactions. Experimental results demonstrate consistent performance improvements over existing transformer-based decoders, with our approach surpassing traditional belief propagation decoders for short-to-medium length LDPC codes.