# Deep Reinforcement Learning-based Rebalancing Policies for Profit Maximization of Relay Nodes in Payment Channel Networks

arXiv: https://arxiv.org/abs/2210.07302 · HTML: https://mlsanity.com/paper/2210.07302/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 49 references that are in the map.
- First posted on arXiv 13 Oct 2022.

## Builds on (most cited first, 13 of 13)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [Benchmarking Deep Reinforcement Learning for Continuous Control](https://mlsanity.com/paper/1604.06778/) (2016)
- [Learning Model Predictive Control for iterative tasks. A Data-Driven Control Framework](https://mlsanity.com/paper/1609.01387/) (2016)
- [A Cryptoeconomic Traffic Analysis of Bitcoin's Lightning Network](https://mlsanity.com/paper/1911.09432/) (2019)
- [How to profit from payments channels](https://mlsanity.com/paper/1911.08803/) (2019)
- [Throughput Optimal Routing in Blockchain Based Payment Systems](https://mlsanity.com/paper/2001.05299/) (2019)
- [Everything is a Race and Nakamoto Always Wins](https://mlsanity.com/paper/2005.10484/) (2020)
- [Imbalance measure and proactive channel rebalancing algorithm for the Lightning Network](https://mlsanity.com/paper/1912.09555/) (2019)
- [The Merchant: Avoiding Payment Channel Depletion through Incentives](https://mlsanity.com/paper/2012.10280/) (2020)
- [Understanding the Benefit of Being Patient in Payment Channel Networks](https://mlsanity.com/paper/2104.02936/) (2021)
- [HIDE & SEEK: Privacy-Preserving Rebalancing on Payment Channel Networks](https://mlsanity.com/paper/2110.08848/) (2021)
- [Online Admission Control and Rebalancing in Payment Channel Networks](https://mlsanity.com/paper/2209.11936/) (2022)

## Abstract (arXiv)

Payment channel networks (PCNs) are a layer-2 blockchain scalability solution, with its main entity, the payment channel, enabling transactions between pairs of nodes "off-chain," thus reducing the burden on the layer-1 network. Nodes with multiple channels can serve as relays for multihop payments by providing their liquidity and withholding part of the payment amount as a fee. Relay nodes might after a while end up with one or more unbalanced channels, and thus need to trigger a rebalancing operation. In this paper, we study how a relay node can maximize its profits from fees by using the rebalancing method of submarine swaps. We introduce a stochastic model to capture the dynamics of a relay node observing random transaction arrivals and performing occasional rebalancing operations, and express the system evolution as a Markov Decision Process. We formulate the problem of the maximization of the node's fortune over time over all rebalancing policies, and approximate the optimal solution by designing a Deep Reinforcement Learning (DRL)-based rebalancing policy. We build a discrete event simulator of the system and use it to demonstrate the DRL policy's superior performance under most conditions by conducting a comparative study of different policies and parameterizations. Our work is the first to introduce DRL for liquidity management in the complex world of PCNs.