# Digital Twin-Aided Learning for Managing Reconfigurable Intelligent Surface-Assisted, Uplink, User-Centric Cell-Free Systems

arXiv: https://arxiv.org/abs/2302.05073 · HTML: https://mlsanity.com/paper/2302.05073/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 71 references that are in the map.
- First posted on arXiv 10 Feb 2023.

## Builds on (most cited first, 23 of 23)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Applications of Deep Reinforcement Learning in Communications and Networking: A Survey](https://mlsanity.com/paper/1810.07862/) (2018)
- [Global Optimality Guarantees For Policy Gradient Methods](https://mlsanity.com/paper/1906.01786/) (2019)
- [Reconfigurable Intelligent Surface Assisted Multiuser MISO Systems Exploiting Deep Reinforcement Learning](https://mlsanity.com/paper/2002.10072/) (2020)
- [Towards Smart and Reconfigurable Environment: Intelligent Reflecting Surface Aided Wireless Network](https://mlsanity.com/paper/1905.00152/) (2019)
- [Intelligent Reflecting Surface: Practical Phase Shift Model and Beamforming Optimization](https://mlsanity.com/paper/1907.06002/) (2019)
- [On-board Deep Q-Network for UAV-assisted Online Power Transfer and Data Collection](https://mlsanity.com/paper/1906.07064/) (2019)
- [The Paradigm of Digital Twin Communications](https://mlsanity.com/paper/2105.07182/) (2021)
- [Intelligent Reflecting Surface: Practical Phase Shift Model and Beamforming Optimization](https://mlsanity.com/paper/2002.10112/) (2020)
- [A joint precoding framework for wideband reconfigurable intelligent surface-aided cell-free network](https://mlsanity.com/paper/2002.03744/) (2020)
- [Channel Estimation for RIS-Aided Multiuser Millimeter-Wave Systems](https://mlsanity.com/paper/2106.14792/) (2021)
- [Energy Efficiency Maximization in RIS-Aided Cell-Free Network with Limited Backhaul](https://mlsanity.com/paper/2012.11736/) (2020)
- [User-Centric 5G Cellular Networks: Resource Allocation and Comparison with the Cell-Free Massive MIMO Approach](https://mlsanity.com/paper/1803.02261/) (2018)
- [Energy-Efficient Power Control in Cell-Free and User-Centric Massive MIMO at Millimeter Wave](https://mlsanity.com/paper/1903.11365/) (2019)
- [Decentralized Beamforming Design for Intelligent Reflecting Surface-enhanced Cell-free Networks](https://mlsanity.com/paper/2006.12238/) (2020)
- [A Digital Twin for Reconfigurable Intelligent Surface Assisted Wireless Communication](https://mlsanity.com/paper/2009.00454/) (2020)
- [Beyond Cell-free MIMO: Energy Efficient Reconfigurable Intelligent Surface Aided Cell-free MIMO Communications](https://mlsanity.com/paper/2011.08473/) (2020)
- [Joint Location and Beamforming Design for STAR-RIS Assisted NOMA Systems](https://mlsanity.com/paper/2206.12836/) (2022)
- [RIS-Aided MIMO Systems with Hardware Impairments: Robust Beamforming Design and Analysis](https://mlsanity.com/paper/2209.11425/) (2022)
- [Multi-Active Multi-Passive (MAMP)-IRS Aided Wireless Communication: A Multi-Hop Beam Routing Design](https://mlsanity.com/paper/2209.06390/) (2022)
- [Two-Timescale Transmission Design for RIS-Aided Cell-Free Massive MIMO Systems](https://mlsanity.com/paper/2210.08514/) (2022)

## Built on by (most cited first, 3 of 3)

- [AI Simulation by Digital Twins: Systematic Survey, Reference Framework, and Mapping to a Standardized Architecture](https://mlsanity.com/paper/2506.06580/) (2025)
- [Multiverse at the Edge: Interacting Real World and Digital Twins for Wireless Beamforming](https://mlsanity.com/paper/2305.10350/) (2023)
- [Developing AI Agents with Simulated Data: Why, what, and how?](https://mlsanity.com/paper/2602.15816/) (2026)

## Abstract (arXiv)

This paper puts forth a new, reconfigurable intelligent surface (RIS)-assisted, uplink, user-centric cell-free (UCCF) system managed with the assistance of a digital twin (DT). Specifically, we propose a novel learning framework that maximizes the sum-rate by jointly optimizing the access point and user association (AUA), power control, and RIS beamforming. This problem is challenging and has never been addressed due to its prohibitively large and complex solution space. Our framework decouples the AUA from the power control and RIS beamforming (PCRB) based on the different natures of their variables, hence reducing the solution space. A new position-adaptive binary particle swarm optimization (PABPSO) method is designed for the AUA. Two twin-delayed deep deterministic policy gradient (TD3) models with new and refined state pre-processing layers are developed for the PCRB. Another important aspect is that a DT is leveraged to train the learning framework with its replay of channel estimates stored. The AUA, power control, and RIS beamforming are only tested in the physical environment at the end of selected epochs. Simulations show that using RISs contributes to considerable increases in the sum-rate of UCCF systems, and the DT dramatically reduces overhead with marginal performance loss. The proposed framework is superior to its alternatives in terms of sum-rate and convergence stability.