# Deep Learning-based List Sphere Decoding for Faster-than-Nyquist (FTN) Signaling Detection

arXiv: https://arxiv.org/abs/2204.07569 · HTML: https://mlsanity.com/paper/2204.07569/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 20 references that are in the map.
- First posted on arXiv 15 Apr 2022.

## Builds on (most cited first, 4 of 4)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Very Brief Introduction to Machine Learning With Applications to Communication Systems](https://mlsanity.com/paper/1808.02342/) (2018)
- [Deep Learning Based Sphere Decoding](https://mlsanity.com/paper/1807.03162/) (2018)
- [Low-Complexity Detection for Faster-than-Nyquist Signaling based on Probabilistic Data Association](https://mlsanity.com/paper/1912.10315/) (2019)

## Built on by (most cited first, 1 of 1)

- [Low Complexity Classification Approach for Faster-than-Nyquist (FTN) Signalling Detection](https://mlsanity.com/paper/2208.10637/) (2022)

## Abstract (arXiv)

Faster-than-Nyquist (FTN) signaling is a candidate non-orthonormal transmission technique to improve the spectral efficiency (SE) of future communication systems. However, such improvements of the SE are at the cost of additional computational complexity to remove the intentionally introduced intersymbol interference. In this paper, we investigate the use of deep learning (DL) to reduce the detection complexity of FTN signaling. To eliminate the need of having a noise whitening filter at the receiver, we first present an equivalent FTN signaling model based on using a set of orthonormal basis functions and identify its operation region. Second, we propose a DL-based list sphere decoding (DL-LSD) algorithm that selects and updates the initial radius of the original LSD to guarantee a pre-defined number NL of lattice points inside the hypersphere. This is achieved by training a neural network to output an approximate initial radius that includes NL lattice points. At the testing phase, if the hypersphere has more than NL lattice points, we keep the NL closest points to the point corresponding to the received FTN signal; however, if the hypersphere has less than NL points, we increase the approximate initial radius by a value that depends on the standard deviation of the distribution of the output radii from the training phase. Then, the approximate value of the log-likelihood ratio (LLR) is calculated based on the obtained NL points. Simulation results show that the computational complexity of the proposed DL-LSD is lower than its counterpart of the original LSD by orders of magnitude.