Physical Layer Authentication and Security Design in the Machine Learning Era

Security at the physical layer (PHY) is a salient research topic in wireless systems, and machine learning (ML) is emerging as a powerful tool for providing new data-driven security solutions. Therefore, the application of ML techniques to the PHY security is of crucial importance in the landscape of more and more data-driven wireless services. In this context, we first summarize the family of bespoke ML algorithms that are eminently suitable for wireless security. Then, we review the recent progress in ML-aided PHY security, where the term "PHY security" is classified into two different types: i) PHY authentication and ii) secure PHY transmission. Moreover, we treat neural networks as special types of ML and present how to deal with PHY security optimization problems using neural networks. Finally, we identify some major challenges and opportunities in tackling PHY security challenges by applying carefully tailored ML tools.