# Deep Reinforcement Learning Based Joint Downlink Beamforming and RIS Configuration in RIS-aided MU-MISO Systems Under Hardware Impairments and Imperfect CSI

arXiv: https://arxiv.org/abs/2211.09702 · HTML: https://mlsanity.com/paper/2211.09702/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 35 references that are in the map.
- First posted on arXiv 10 Oct 2022.

## Builds on (most cited first, 19 of 19)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [Deep Reinforcement Learning that Matters](https://mlsanity.com/paper/1709.06560/) (2017)
- [Intelligent Reflecting Surface Enhanced Wireless Network via Joint Active and Passive Beamforming](https://mlsanity.com/paper/1810.03961/) (2018)
- [Reconfigurable Intelligent Surface Assisted Multiuser MISO Systems Exploiting Deep Reinforcement Learning](https://mlsanity.com/paper/2002.10072/) (2020)
- [Deep Reinforcement Learning Based Intelligent Reflecting Surface for Secure Wireless Communications](https://mlsanity.com/paper/2002.12271/) (2020)
- [Intelligent Reflecting Surface: Practical Phase Shift Model and Beamforming Optimization](https://mlsanity.com/paper/2002.10112/) (2020)
- [Machine Learning Empowered Trajectory and Passive Beamforming Design in UAV-RIS Wireless Networks](https://mlsanity.com/paper/2010.02749/) (2020)
- [Optimizing Age of Information Through Aerial Reconfigurable Intelligent Surfaces: A Deep Reinforcement Learning Approach](https://mlsanity.com/paper/2011.04817/) (2020)
- [RIS Enhanced Massive Non-orthogonal Multiple Access Networks: Deployment and Passive Beamforming Design](https://mlsanity.com/paper/2001.10363/) (2020)
- [Reconfigurable Intelligent Surface Enabled Vehicular Communication: Joint User Scheduling and Passive Beamforming](https://mlsanity.com/paper/2101.12247/) (2021)
- [Hybrid Beamforming for RIS-Empowered Multi-hop Terahertz Communications: A DRL-based Method](https://mlsanity.com/paper/2009.09380/) (2020)
- [Deep Intrinsically Motivated Exploration in Continuous Control](https://mlsanity.com/paper/2210.00293/) (2022)
- [Millimeter Wave Communications with an Intelligent Reflector: Performance Optimization and Distributional Reinforcement Learning](https://mlsanity.com/paper/2002.10572/) (2020)
- [On the Impact of Hardware Impairments on RIS-aided Localization](https://mlsanity.com/paper/2205.12599/) (2022)

## Built on by (most cited first, 1 of 1)

- [Fairness-Driven Optimization of RIS-Augmented 5G Networks for Seamless 3D UAV Connectivity Using DRL Algorithms](https://mlsanity.com/paper/2312.09420/) (2023)

## Abstract (arXiv)

We introduce a novel deep reinforcement learning (DRL) approach to jointly optimize transmit beamforming and reconfigurable intelligent surface (RIS) phase shifts in a multiuser multiple input single output (MU-MISO) system to maximize the sum downlink rate under the phase-dependent reflection amplitude model. Our approach addresses the challenge of imperfect channel state information (CSI) and hardware impairments by considering a practical RIS amplitude model. We compare the performance of our approach against a vanilla DRL agent in two scenarios: perfect CSI and phase-dependent RIS amplitudes, and mismatched CSI and ideal RIS reflections. The results demonstrate that the proposed framework significantly outperforms the vanilla DRL agent under mismatch and approaches the golden standard. Our contributions include modifications to the DRL approach to address the joint design of transmit beamforming and phase shifts and the phase-dependent amplitude model. To the best of our knowledge, our method is the first DRL-based approach for the phase-dependent reflection amplitude model in RIS-aided MU-MISO systems. Our findings in this study highlight the potential of our approach as a promising solution to overcome hardware impairments in RIS-aided wireless communication systems.