# Ultra-Reliable Indoor Millimeter Wave Communications using Multiple Artificial Intelligence-Powered Intelligent Surfaces

arXiv: https://arxiv.org/abs/2104.00075 · HTML: https://mlsanity.com/paper/2104.00075/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 48 references that are in the map.
- First posted on arXiv 31 Mar 2021.

## Builds on (most cited first, 17 of 17)

- [Deep Recurrent Q-Learning for Partially Observable MDPs](https://mlsanity.com/paper/1507.06527/) (2015)
- [A Vision of 6G Wireless Systems: Applications, Trends, Technologies, and Open Research Problems](https://mlsanity.com/paper/1902.10265/) (2019)
- [Risk-sensitive Reinforcement Learning](https://mlsanity.com/paper/1311.2097/) (2013)
- [Wireless Network Intelligence at the Edge](https://mlsanity.com/paper/1812.02858/) (2018)
- [Intelligent Reflecting Surface Enhanced Wireless Network via Joint Active and Passive Beamforming](https://mlsanity.com/paper/1810.03961/) (2018)
- [Reconfigurable Intelligent Surface Assisted Multiuser MISO Systems Exploiting Deep Reinforcement Learning](https://mlsanity.com/paper/2002.10072/) (2020)
- [Wireless Communications Through Reconfigurable Intelligent Surfaces](https://mlsanity.com/paper/1906.09490/) (2019)
- [Reconfigurable Intelligent Surfaces: Principles and Opportunities](https://mlsanity.com/paper/2007.03435/) (2020)
- [Indoor Signal Focusing with Deep Learning Designed Reconfigurable Intelligent Surfaces](https://mlsanity.com/paper/1905.07726/) (2019)
- [Beamforming Optimization for Wireless Network Aided by Intelligent Reflecting Surface with Discrete Phase Shifts](https://mlsanity.com/paper/1906.03165/) (2019)
- [Unsupervised Learning for Passive Beamforming](https://mlsanity.com/paper/2001.02348/) (2020)
- [Deep Reinforcement Learning for Intelligent Reflecting Surfaces: Towards Standalone Operation](https://mlsanity.com/paper/2002.11101/) (2020)
- [Reflections in the Sky: Millimeter Wave Communication with UAV-Carried Intelligent Reflectors](https://mlsanity.com/paper/1908.03271/) (2019)
- [Risk-Based Optimization of Virtual Reality over Terahertz Reconfigurable Intelligent Surfaces](https://mlsanity.com/paper/2002.09052/) (2020)
- [Channel Estimation Method and Phase Shift Design for Reconfigurable Intelligent Surface Assisted MIMO Networks](https://mlsanity.com/paper/1912.10671/) (2019)
- [Realizing Wireless Communication through Software-defined HyperSurface Environments](https://mlsanity.com/paper/1805.06677/) (2018)
- [Optimized Deployment of Millimeter Wave Networks for In-venue Regions with Stochastic Users' Orientation](https://mlsanity.com/paper/1906.02491/) (2019)

## Built on by (most cited first, 2 of 2)

- [Risk-Sensitive Multi-Agent Reinforcement Learning in Network Aggregative Markov Games](https://mlsanity.com/paper/2402.05906/) (2024)
- [Risk Sensitivity in Markov Games and Multi-Agent Reinforcement Learning: A Systematic Review](https://mlsanity.com/paper/2406.06041/) (2024)

## Abstract (arXiv)

In this paper, a novel framework for guaranteeing ultra-reliable millimeter wave (mmW) communications using multiple artificial intelligence (AI)-enabled reconfigurable intelligent surfaces (RISs) is proposed. The use of multiple AI-powered RISs allows changing the propagation direction of the signals transmitted from a mmW access point (AP) thereby improving coverage particularly for non-line-of-sight (NLoS) areas. However, due to the possibility of highly stochastic blockage over mmW links, designing an intelligent controller to jointly optimize the mmW AP beam and RIS phase shifts is a daunting task. In this regard, first, a parametric risk-sensitive episodic return is proposed to maximize the expected bit rate and mitigate the risk of mmW link blockage. Then, a closed-form approximation of the policy gradient of the risk-sensitive episodic return is analytically derived. Next, the problem of joint beamforming for mmW AP and phase shift control for mmW RISs is modeled as an identical payoff stochastic game within a cooperative multi-agent environment, in which the agents are the mmW AP and the RISs. Two centralized and distributed controllers are proposed to control the policies of the mmW AP and RISs. To directly find an optimal solution, the parametric functional-form policies for these controllers are modeled using deep recurrent neural networks (RNNs). Simulation results show that the error between policies of the optimal and the RNN-based controllers is less than 1.5%. Moreover, the variance of the achievable rates resulting from the deep RNN-based controllers is 60% less than the variance of the risk-averse baseline.