# Joint Deep Reinforcement Learning and Unfolding: Beam Selection and Precoding for mmWave Multiuser MIMO with Lens Arrays

arXiv: https://arxiv.org/abs/2101.01336 · HTML: https://mlsanity.com/paper/2101.01336/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 38 references that are in the map.
- First posted on arXiv 5 Jan 2021.

## Builds on (most cited first, 11 of 11)

- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Learning to Optimize: Training Deep Neural Networks for Wireless Resource Management](https://mlsanity.com/paper/1705.09412/) (2017)
- [Multi-Agent Reinforcement Learning Based Resource Allocation for UAV Networks](https://mlsanity.com/paper/1810.10408/) (2018)
- [Deep Unfolding for Communications Systems: A Survey and Some New Directions](https://mlsanity.com/paper/1906.05774/) (2019)
- [Iterative Algorithm Induced Deep-Unfolding Neural Networks: Precoding Design for Multiuser MIMO Systems](https://mlsanity.com/paper/2006.08099/) (2020)
- [Deep Reinforcement Learning for Dynamic Multichannel Access in Wireless Networks](https://mlsanity.com/paper/1802.06958/) (2018)
- [Five Disruptive Technology Directions for 5G](https://mlsanity.com/paper/1312.0229/) (2013)
- [Deep Reinforcement Learning Based Mode Selection and Resource Management for Green Fog Radio Access Networks](https://mlsanity.com/paper/1809.05629/) (2018)
- [RIS Enhanced Massive Non-orthogonal Multiple Access Networks: Deployment and Passive Beamforming Design](https://mlsanity.com/paper/2001.10363/) (2020)
- [FALP: Fast beam alignment in mmWave systems with low-resolution phase shifters](https://mlsanity.com/paper/1902.05714/) (2019)
- [Beamspace Precoding and Beam Selection for Wideband Millimeter-Wave MIMO Relying on Lens Antenna Arrays](https://mlsanity.com/paper/2004.08071/) (2020)

## Built on by (most cited first, 6 of 6)

- [Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applications](https://mlsanity.com/paper/2111.12444/) (2021)
- [Two-Timescale End-to-End Learning for Channel Acquisition and Hybrid Precoding](https://mlsanity.com/paper/2110.12059/) (2021)
- [DDPG-Driven Deep-Unfolding with Adaptive Depth for Channel Estimation with Sparse Bayesian Learning](https://mlsanity.com/paper/2201.08477/) (2022)
- [Deep-Unfolding for Next-Generation Transceivers](https://mlsanity.com/paper/2305.08303/) (2023)
- [Efficient Beam Selection for ISAC in Cell-Free Massive MIMO via Digital Twin-Assisted Deep Reinforcement Learning](https://mlsanity.com/paper/2506.18560/) (2025)
- [Environment-Aware Transfer Reinforcement Learning for Sustainable Beam Selection](https://mlsanity.com/paper/2511.11647/) (2025)

## Abstract (arXiv)

The millimeter wave (mmWave) multiuser multiple-input multiple-output (MU-MIMO) systems with discrete lens arrays (DLA) have received great attention due to their simple hardware implementation and excellent performance. In this work, we investigate the joint design of beam selection and digital precoding matrices for mmWave MU-MIMO systems with DLA to maximize the sum-rate subject to the transmit power constraint and the constraints of the selection matrix structure. The investigated non-convex problem with discrete variables and coupled constraints is challenging to solve and an efficient framework of joint neural network (NN) design is proposed to tackle it. Specifically, the proposed framework consists of a deep reinforcement learning (DRL)-based NN and a deep-unfolding NN, which are employed to optimize the beam selection and digital precoding matrices, respectively. As for the DRL-based NN, we formulate the beam selection problem as a Markov decision process and a double deep Q-network algorithm is developed to solve it. The base station is considered to be an agent, where the state, action, and reward function are carefully designed. Regarding the design of the digital precoding matrix, we develop an iterative weighted minimum mean-square error algorithm induced deep-unfolding NN, which unfolds this algorithm into a layerwise structure with introduced trainable parameters. Simulation results verify that this jointly trained NN remarkably outperforms the existing iterative algorithms with reduced complexity and stronger robustness.