mlsanityData to 7 Oct 2026

Reducing Computational Complexity of Neural Networks in Optical Channel Equalization: From Concepts to Implementation

arXiv 2208.12866 ↗First posted 26 Aug 2022eess.SP cs.CC cs.ETcited by 6

Not on the map. Why?

Abstract (arXiv)

In this paper, a new methodology is proposed that allows for the low-complexity development of neural network (NN) based equalizers for the mitigation of impairments in high-speed coherent optical transmission systems. In this work, we provide a comprehensive description and comparison of various deep model compression approaches that have been applied to feed-forward and recurrent NN designs. Additionally, we evaluate the influence these strategies have on the performance of each NN equalizer. Quantization, weight clustering, pruning, and other cutting-edge strategies for model compression are taken into consideration. In this work, we propose and evaluate a Bayesian optimization-assisted compression, in which the hyperparameters of the compression are chosen to simultaneously reduce complexity and improve performance. In conclusion, the trade-off between the complexity of each compression approach and its performance is evaluated by utilizing both simulated and experimental data in order to complete the analysis. By utilizing optimal compression approaches, we show that it is possible to design an NN-based equalizer that is simpler to implement and has better performance than the conventional digital back-propagation (DBP) equalizer with only one step per span. This is accomplished by reducing the number of multipliers used in the NN equalizer after applying the weighted clustering and pruning algorithms. Furthermore, we demonstrate that an equalizer based on NN can also achieve superior performance while still maintaining the same degree of complexity as the full electronic chromatic dispersion compensation block. We conclude our analysis by highlighting open questions and existing challenges, as well as possible future research directions.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 53 of its 139 references that are in the map. First posted on arXiv 26 Aug 2022.

Builds on

53 of 139 references.

References · 53

  1. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  2. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  3. 2020Knowledge Distillation: A Surveycited by 1,578
  4. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  5. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  6. 2018A Tutorial on Bayesian Optimizationcited by 734
  7. 2017To prune, or not to prune: exploring the efficacy of pruning for model compressioncited by 596
  8. 2018Rethinking the Value of Network Pruningcited by 595
  9. 2020What is the State of Neural Network Pruning?cited by 578
  10. 2018AMC: AutoML for Model Compression and Acceleration on Mobile Devicescited by 375
  11. 2017A Survey of Model Compression and Acceleration for Deep Neural Networkscited by 298
  12. 2021Pruning and Quantization for Deep Neural Network Acceleration: A Surveycited by 259
38 more
  1. 2020Comparing Rewinding and Fine-tuning in Neural Network Pruningcited by 238
  2. 2019MetaPruning: Meta Learning for Automatic Neural Network Channel Pruningcited by 226
  3. 2017Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weightscited by 211
  4. 2020Compressing BERT: Studying the Effects of Weight Pruning on Transfer Learningcited by 184
  5. 2019Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networkscited by 143
  6. 2018Fast inference of deep neural networks in FPGAs for particle physicscited by 132
  7. 2016Convolutional Neural Networks using Logarithmic Data Representationcited by 97
  8. 2020Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectorscited by 78
  9. 2017Training Quantized Nets: A Deeper Understandingcited by 74
  10. 2019DeepShift: Towards Multiplication-Less Neural Networkscited by 52
  11. 2020ShiftAddNet: A Hardware-Inspired Deep Networkcited by 52
  12. 2020Methods for Pruning Deep Neural Networkscited by 49
  13. 2021Towards Efficient Post-training Quantization of Pre-trained Language Modelscited by 37
  14. 2019Training Quantized Neural Networks with a Full-precision Auxiliary Modulecited by 35
  15. 2018Deep k-Means: Re-Training and Parameter Sharing with Harder Cluster Assignments for Compressing Deep Convolutionscited by 33
  16. 2021Ps and Qs: Quantization-aware pruning for efficient low latency neural network inferencecited by 31
  17. 2021DKM: Differentiable K-Means Clustering Layer for Neural Network Compressioncited by 31
  18. 2022Power-of-Two Quantization for Low Bitwidth and Hardware Compliant Neural Networkscited by 26
  19. 2021TransTailor: Pruning the Pre-trained Model for Improved Transfer Learningcited by 24
  20. 2018An Overview on Application of Machine Learning Techniques in Optical Networkscited by 19
  21. 2020Importance of Tuning Hyperparameters of Machine Learning Algorithmscited by 19
  22. 2021Performance versus Complexity Study of Neural Network Equalizers in Coherent Optical Systemscited by 17
  23. 2016On the efficient representation and execution of deep acoustic modelscited by 13
  24. 2017Nonlinear Interference Mitigation via Deep Neural Networkscited by 13
  25. 2019Bit Efficient Quantization for Deep Neural Networkscited by 13
  26. 2021Neural Network Quantization for Efficient Inference: A Surveycited by 13
  27. 2021Cluster-Promoting Quantization with Bit-Drop for Minimizing Network Quantization Losscited by 11
  28. 2020Physics-Based Deep Learning for Fiber-Optic Communication Systemscited by 9
  29. 2021HPTQ: Hardware-Friendly Post Training Quantizationcited by 9
  30. 2020Compensation of Fiber Nonlinearities in Digital Coherent Systems Leveraging Long Short-Term Memory Neural Networkscited by 8
  31. 2021Fixed-point Quantization of Convolutional Neural Networks for Quantized Inference on Embedded Platformscited by 7
  32. 2021Transfer Learning for Neural Networks-based Equalizers in Coherent Optical Systemscited by 7
  33. 2021Performance and Complexity Analysis of bi-directional Recurrent Neural Network Models vs. Volterra Nonlinear Equalizers in Digital Coherent Systemscited by 6
  34. 2021Training Quantized Neural Networks to Global Optimality via Semidefinite Programmingcited by 6
  35. 2021Training Deep Neural Networks with Joint Quantization and Pruning of Weights and Activationscited by 6
  36. 2022PAC-Net: A Model Pruning Approach to Inductive Transfer Learningcited by 6
  37. 2021Neural Networks-based Equalizers for Coherent Optical Transmission: Caveats and Pitfallscited by 5
  38. 2022Domain Adaptation: the Key Enabler of Neural Network Equalizers in Coherent Optical Systemscited by 4

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 2; 2024: 2.

6 papers.

  1. 2023Efficient Neural Networks for Tiny Machine Learning: A Comprehensive Reviewcited by 9
  2. 2022Knowledge Distillation Applied to Optical Channel Equalization: Solving the Parallelization Problem of Recurrent Connectioncited by 3
  3. 2022Implementing Neural Network-Based Equalizers in a Coherent Optical Transmission System Using Field-Programmable Gate Arrayscited by 1
  4. 2023Application of Transformers for Nonlinear Channel Compensation in Optical Systemscited by 1
  5. 2024Artificial Neural Networks for Photonic Applications: From Algorithms to Implementationcited by 0
  6. 2024Error Diffusion: Post Training Quantization with Block-Scaled Number Formats for Neural Networkscited by 0