# Pervasive Machine Learning for Smart Radio Environments Enabled by Reconfigurable Intelligent Surfaces

arXiv: https://arxiv.org/abs/2205.03793 · HTML: https://mlsanity.com/paper/2205.03793/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 109 of its 198 references that are in the map.
- First posted on arXiv 8 May 2022.

## Builds on (most cited first, 30 of 109)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Dota 2 with Large Scale Deep Reinforcement Learning](https://mlsanity.com/paper/1912.06680/) (2019)
- [Recurrent World Models Facilitate Policy Evolution](https://mlsanity.com/paper/1809.01999/) (2018)
- [Solving Rubik's Cube with a Robot Hand](https://mlsanity.com/paper/1910.07113/) (2019)
- [Generating Sequences With Recurrent Neural Networks](https://mlsanity.com/paper/1308.0850/) (2013)
- [Distributional Reinforcement Learning with Quantile Regression](https://mlsanity.com/paper/1710.10044/) (2017)
- [A Theoretical Analysis of Deep Q-Learning](https://mlsanity.com/paper/1901.00137/) (2019)
- [Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Sampling](https://mlsanity.com/paper/1802.09127/) (2018)
- [Deep Reinforcement Learning in Large Discrete Action Spaces](https://mlsanity.com/paper/1512.07679/) (2015)
- [A Vision of 6G Wireless Systems: Applications, Trends, Technologies, and Open Research Problems](https://mlsanity.com/paper/1902.10265/) (2019)
- [6G Networks: Beyond Shannon Towards Semantic and Goal-Oriented Communications](https://mlsanity.com/paper/2011.14844/) (2020)
- [Action Branching Architectures for Deep Reinforcement Learning](https://mlsanity.com/paper/1711.08946/) (2017)
- [Reconfigurable Intelligent Surface Assisted Multiuser MISO Systems Exploiting Deep Reinforcement Learning](https://mlsanity.com/paper/2002.10072/) (2020)
- [Wireless Networks Design in the Era of Deep Learning: Model-Based, AI-Based, or Both?](https://mlsanity.com/paper/1902.02647/) (2019)
- [Towards Smart and Reconfigurable Environment: Intelligent Reflecting Surface Aided Wireless Network](https://mlsanity.com/paper/1905.00152/) (2019)
- [Wireless Communications Through Reconfigurable Intelligent Surfaces](https://mlsanity.com/paper/1906.09490/) (2019)
- [A Tutorial on Beam Management for 3GPP NR at mmWave Frequencies](https://mlsanity.com/paper/1804.01908/) (2018)
- [Intelligent Reflecting Surface Aided Wireless Communications: A Tutorial](https://mlsanity.com/paper/2007.02759/) (2020)
- [Deep Reinforcement Learning Based Intelligent Reflecting Surface for Secure Wireless Communications](https://mlsanity.com/paper/2002.12271/) (2020)
- [Enabling Large Intelligent Surfaces with Compressive Sensing and Deep Learning](https://mlsanity.com/paper/1904.10136/) (2019)
- [Holographic MIMO Surfaces for 6G Wireless Networks: Opportunities, Challenges, and Trends](https://mlsanity.com/paper/1911.12296/) (2019)
- [Deep Channel Learning For Large Intelligent Surfaces Aided mm-Wave Massive MIMO Systems](https://mlsanity.com/paper/2001.11085/) (2020)
- [Deep Contextual Multi-armed Bandits](https://mlsanity.com/paper/1807.09809/) (2018)
- [Intelligent Reflecting Surface: Practical Phase Shift Model and Beamforming Optimization](https://mlsanity.com/paper/1907.06002/) (2019)
- [Dynamic Metasurface Antennas for 6G Extreme Massive MIMO Communications](https://mlsanity.com/paper/2006.07838/) (2020)
- [Deep-Learning-based Millimeter-Wave Massive MIMO for Hybrid Precoding](https://mlsanity.com/paper/1901.06537/) (2019)
- [Rayleigh Fading Modeling and Channel Hardening for Reconfigurable Intelligent Surfaces](https://mlsanity.com/paper/2009.04723/) (2020)
- [Indoor Signal Focusing with Deep Learning Designed Reconfigurable Intelligent Surfaces](https://mlsanity.com/paper/1905.07726/) (2019)

## Built on by (most cited first, 12 of 12)

- [Over-the-Air Edge Inference via End-to-End Metasurfaces-Integrated Artificial Neural Networks](https://mlsanity.com/paper/2504.00233/) (2025)
- [Multi-Agent Reinforcement Learning for Offloading Cellular Communications with Cooperating UAVs](https://mlsanity.com/paper/2402.02957/) (2024)
- [Joint Active RIS Configuration and User Power Control for Localization: A Neuroevolution-Based Approach](https://mlsanity.com/paper/2510.13819/) (2025)
- [Lyapunov-Driven Deep Reinforcement Learning for Edge Inference Empowered by Reconfigurable Intelligent Surfaces](https://mlsanity.com/paper/2305.10931/) (2023)
- [Evasive Active Hypothesis Testing with Deep Neuroevolution: The Single- and Multi-Agent Cases](https://mlsanity.com/paper/2403.10112/) (2024)
- [On the Detection of Non-Cooperative RISs: Scan B-Testing via Deep Support Vector Data Description](https://mlsanity.com/paper/2411.03237/) (2024)
- [Robust Millimeter Beamforming via Self-Supervised Hybrid Deep Learning](https://mlsanity.com/paper/2303.12653/) (2023)
- [Federated Learning Games for Reconfigurable Intelligent Surfaces via Causal Representations](https://mlsanity.com/paper/2306.01306/) (2023)
- [Autoregressive Attention Neural Networks for Non-Line-of-Sight User Tracking with Dynamic Metasurface Antennas](https://mlsanity.com/paper/2310.19767/) (2023)
- [RIS-Enabled Smart Wireless Environments: Fundamentals and Distributed Optimization](https://mlsanity.com/paper/2512.18788/) (2025)
- [Learning to Reflect: Hierarchical Multi-Agent Reinforcement Learning for CSI-Free mmWave Beam-Focusing](https://mlsanity.com/paper/2603.07370/) (2026)
- [Active Sensing for RIS-Aided Tracking and Power Control: A Hybrid Neuroevolution and Supervised Learning Approach](https://mlsanity.com/paper/2607.00056/) (2026)

## Abstract (arXiv)

The emerging technology of Reconfigurable Intelligent Surfaces (RISs) is provisioned as an enabler of smart wireless environments, offering a highly scalable, low-cost, hardware-efficient, and almost energy-neutral solution for dynamic control of the propagation of electromagnetic signals over the wireless medium, ultimately providing increased environmental intelligence for diverse operation objectives. One of the major challenges with the envisioned dense deployment of RISs in such reconfigurable radio environments is the efficient configuration of multiple metasurfaces with limited, or even the absence of, computing hardware. In this paper, we consider multi-user and multi-RIS-empowered wireless systems, and present a thorough survey of the online machine learning approaches for the orchestration of their various tunable components. Focusing on the sum-rate maximization as a representative design objective, we present a comprehensive problem formulation based on Deep Reinforcement Learning (DRL). We detail the correspondences among the parameters of the wireless system and the DRL terminology, and devise generic algorithmic steps for the artificial neural network training and deployment, while discussing their implementation details. Further practical considerations for multi-RIS-empowered wireless communications in the sixth Generation (6G) era are presented along with some key open research challenges. Differently from the DRL-based status quo, we leverage the independence between the configuration of the system design parameters and the future states of the wireless environment, and present efficient multi-armed bandits approaches, whose resulting sum-rate performances are numerically shown to outperform random configurations, while being sufficiently close to the conventional Deep Q-Network (DQN) algorithm, but with lower implementation complexity.