# Learning to Focus: CSI-Free Hierarchical MARL for Reconfigurable Reflectors

arXiv: https://arxiv.org/abs/2604.05165 · HTML: https://mlsanity.com/paper/2604.05165/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 19 references that are in the map.
- First posted on arXiv 6 Apr 2026.

## Builds on (most cited first, 10 of 10)

- [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021)
- [Reconfigurable Intelligent Surface Assisted Multiuser MISO Systems Exploiting Deep Reinforcement Learning](https://mlsanity.com/paper/2002.10072/) (2020)
- [Enabling Large Intelligent Surfaces with Compressive Sensing and Deep Learning](https://mlsanity.com/paper/1904.10136/) (2019)
- [Deep Reinforcement Learning for Intelligent Reflecting Surfaces: Towards Standalone Operation](https://mlsanity.com/paper/2002.11101/) (2020)
- [Two-Timescale Channel Estimation for Reconfigurable Intelligent Surface Aided Wireless Communications](https://mlsanity.com/paper/1912.07990/) (2019)
- [Coverage Enhancement for NLOS mmWave Links Using Passive Reflectors](https://mlsanity.com/paper/1905.04794/) (2019)
- [A Deep Reinforcement Learning Approach for Autonomous Reconfigurable Intelligent Surfaces](https://mlsanity.com/paper/2403.09270/) (2024)
- [Insight Gained from Migrating a Machine Learning Model to Intelligence Processing Units](https://mlsanity.com/paper/2404.10730/) (2024)
- [Guiding Wireless Signals with Arrays of Metallic Linear Fresnel Reflectors: A Low-cost, Frequency-versatile, and Practical Approach](https://mlsanity.com/paper/2407.19179/) (2024)
- [Signal Whisperers: Enhancing Wireless Reception Using DRL-Guided Reflector Arrays](https://mlsanity.com/paper/2501.15044/) (2025)

## Abstract (arXiv)

Reconfigurable Intelligent Surfaces (RIS) have the potential to engineer smart radio environments for next-generation millimeter-wave (mmWave) networks. However, the prohibitive computational overhead of Channel State Information (CSI) estimation and the dimensionality explosion inherent in centralized optimization severely hinder practical large-scale deployments. To overcome these bottlenecks, we introduce a per-element CSI-free paradigm powered by a Hierarchical Multi-Agent Reinforcement Learning (HMARL) architecture to control mechanically reconfigurable reflective surfaces. By substituting pilot-based channel estimation for each element of the device with accessible user localization data, our framework leverages spatial intelligence for macro-scale wave propagation management. The control problem is decomposed into a two-tier neural architecture: a high-level controller executes temporally extended, discrete user-to-reflector allocations, while low-level controllers autonomously optimize continuous focal points using Multi-Agent Proximal Policy Optimization (MAPPO) under a Centralized Training with Decentralized Execution (CTDE) scheme. Comprehensive deterministic ray-tracing evaluations in an indoor mmWave scenario demonstrate that this hierarchical framework achieves received signal strength indicator (RSSI) improvements of up to 7.79 dB over centralized Proximal Policy Optimization (PPO) baselines. Furthermore, the system maintains resilient beam-focusing performance under practical sub-meter localization tracking errors for up to four users and two reflector arrays. By eliminating execution-time CSI overhead while preserving high-fidelity signal redirection, this work provides a scalable and cost-effective step toward intelligent indoor wireless environments.