# Deep Reinforcement Learning for IRS Phase Shift Design in Spatiotemporally Correlated Environments

arXiv: https://arxiv.org/abs/2211.09726 · HTML: https://mlsanity.com/paper/2211.09726/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 23 references that are in the map.
- First posted on arXiv 2 Nov 2022.

## Builds on (most cited first, 15 of 15)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Towards Understanding the Spectral Bias of Deep Learning](https://mlsanity.com/paper/1912.01198/) (2019)
- [Intelligent Reflecting Surface Enhanced Wireless Network via Joint Active and Passive Beamforming](https://mlsanity.com/paper/1810.03961/) (2018)
- [Towards Characterizing Divergence in Deep Q-Learning](https://mlsanity.com/paper/1903.08894/) (2019)
- [Reconfigurable Intelligent Surface Assisted Multiuser MISO Systems Exploiting Deep Reinforcement Learning](https://mlsanity.com/paper/2002.10072/) (2020)
- [Smart Radio Environments Empowered by Reconfigurable Intelligent Surfaces: How it Works, State of Research, and Road Ahead](https://mlsanity.com/paper/2004.09352/) (2020)
- [Learning to Reflect and to Beamform for Intelligent Reflecting Surface with Implicit Channel Estimation](https://mlsanity.com/paper/2009.14404/) (2020)
- [Enabling Large Intelligent Surfaces with Compressive Sensing and Deep Learning](https://mlsanity.com/paper/1904.10136/) (2019)
- [Indoor Signal Focusing with Deep Learning Designed Reconfigurable Intelligent Surfaces](https://mlsanity.com/paper/1905.07726/) (2019)
- [Robust and Secure Wireless Communications via Intelligent Reflecting Surfaces](https://mlsanity.com/paper/1912.01497/) (2019)
- [Over-the-Air Computation via Intelligent Reflecting Surfaces](https://mlsanity.com/paper/1904.12475/) (2019)
- [On the Robustness of Deep Reinforcement Learning in IRS-Aided Wireless Communications Systems](https://mlsanity.com/paper/2107.08293/) (2021)
- [Spatially Controlled Relay Beamforming: 2-Stage Optimal Policies](https://mlsanity.com/paper/1705.07463/) (2017)
- [Dual-Function Radar-Communication System Aided by Intelligent Reflecting Surfaces](https://mlsanity.com/paper/2204.04721/) (2022)

## Abstract (arXiv)

The paper studies the problem of designing the Intelligent Reflecting Surface (IRS) phase shifters for Multiple Input Single Output (MISO) communication systems in spatiotemporally correlated channel environments, where the destination can move within a confined area. The objective is to maximize the expected sum of SNRs at the receiver over infinite time horizons. The problem formulation gives rise to a Markov Decision Process (MDP). We propose a deep actor-critic algorithm that accounts for channel correlations and destination motion by constructing the state representation to include the current position of the receiver and the phase shift values and receiver positions that correspond to a window of previous time steps. The channel variability induces high frequency components on the spectrum of the underlying value function. We propose the preprocessing of the critic's input with a Fourier kernel which enables stable value learning. Finally, we investigate the use of the destination SNR as a component of the designed MDP state, which is common practice in previous work. We provide empirical evidence that, when the channels are spatiotemporally correlated, the inclusion of the SNR in the state representation interacts with function approximation in ways that inhibit convergence.