# A Heuristic-Integrated DRL Approach for Phase Optimization in Large-Scale RISs

arXiv: https://arxiv.org/abs/2505.04401 · HTML: https://mlsanity.com/paper/2505.04401/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 18 references that are in the map.
- First posted on arXiv 7 May 2025.

## Builds on (most cited first, 11 of 11)

- [Reconfigurable Intelligent Surfaces: Principles and Opportunities](https://mlsanity.com/paper/2007.03435/) (2020)
- [Rate-Splitting Multiple Access: Fundamentals, Survey, and Future Research Trends](https://mlsanity.com/paper/2201.03192/) (2022)
- [Machine Learning Empowered Trajectory and Passive Beamforming Design in UAV-RIS Wireless Networks](https://mlsanity.com/paper/2010.02749/) (2020)
- [Deep Reinforcement Learning for Optimizing RIS-Assisted HD-FD Wireless Systems](https://mlsanity.com/paper/2110.04859/) (2021)
- [Codebook Design and Beam Training for Extremely Large-Scale RIS: Far-Field or Near-Field?](https://mlsanity.com/paper/2109.10143/) (2021)
- [Heuristic Algorithms for RIS-assisted Wireless Networks: Exploring Heuristic-aided Machine Learning](https://mlsanity.com/paper/2307.01205/) (2023)
- [Aerial STAR-RIS Empowered MEC: A DRL Approach for Energy Minimization](https://mlsanity.com/paper/2312.08714/) (2023)
- [Resource Management for Transmit Power Minimization in UAV-Assisted RIS HetNets Supported by Dual Connectivity](https://mlsanity.com/paper/2106.13174/) (2021)
- [Analysis and Optimization of RIS-Assisted Cell-Free Massive MIMO NOMA Systems](https://mlsanity.com/paper/2407.04006/) (2024)
- [Intelligent Reflecting Surface Configurations for Smart Radio Using Deep Reinforcement Learning](https://mlsanity.com/paper/2205.05269/) (2022)
- [DRL-Based Sidelobe Suppression for Multi-focus Reconfigurable Intelligent Surface](https://mlsanity.com/paper/2312.15565/) (2023)

## Built on by (most cited first, 1 of 1)

- [Optimization Algorithms for Joint OFDM Waveform Design and RIS Configuration in 6G Networks: From Convex Relaxation to Foundation Models](https://mlsanity.com/paper/2606.31334/) (2026)

## Abstract (arXiv)

Optimizing discrete phase shifts in large-scale reconfigurable intelligent surfaces (RISs) is challenging due to their non-convex and non-linear nature. In this letter, we propose a heuristic-integrated deep reinforcement learning (DRL) framework that (1) leverages accumulated actions over multiple steps in the double deep Q-network (DDQN) for RIS column-wise control and (2) integrates a greedy algorithm (GA) into each DRL step to refine the state via fine-grained, element-wise optimization of RIS configurations. By learning from GA-included states, the proposed approach effectively addresses RIS optimization within a small DRL action space, demonstrating its capability to optimize phase-shift configurations of large-scale RISs.