Differentiable Policy Transport over Multi-Layer Network Feasibility Geometry
Not on the map. Why?
Abstract (arXiv)
Learning-based control is increasingly central to automating network operations. A learned policy, however, must satisfy cross-layer constraints on interference, power-rate coupling, flow conservation, service chains, capacity, latency, and reliability. Existing methods typically account for only a subset of this geometry and only indirectly, e.g., through reward penalties, Lagrange multipliers, or post-hoc repairs. This paper proposes Network Feasibility Geometry Reinforcement Learning (NFG-RL), which models coupled constraints via transport theory and the residual inclusion bphi_mathfrakN(x,a)∈cK_mathfrakN, defining the executed policy as the pushforward of a proto-policy through a feasibility-transport map. NFG-RL compiles heterogeneous constraints into typed residual blocks and transports proto-actions through a differentiable variational operator, letting active constraints shape execution, exploration, and actor gradients. Our analysis shows that exact transport yields almost-sure feasible execution, while active constraints contract exploration onto the feasible tangent space. It further establishes a nonnegative first-order gain from critic-tilted transport over plain projection and recovers backpressure scheduling as the gradient of a lifted drift residual. In two public-trace-conditioned wireless-edge surrogate environments, NFG-RL improves feasible utility by 37.5--41.5% over the strongest non-NFG method in each environment, reduces raw-action violation by 48.5--60.8%, and lowers P99 delay by 57.0--75.5%, outperforming a range of optimization and learning baselines.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 33 of its 55 references that are in the map. First posted on arXiv 2 Aug 2026.
Builds on
33 of 55 references.
References · 33
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
- 2021Convex optimizationcited by 2,515
- 2017OptNet: Differentiable Optimization as a Layer in Neural Networkscited by 599
- 2019Differentiable Convex Optimization Layerscited by 465
- 2018Reward Constrained Policy Optimizationcited by 417
- 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 297
- 2018Safe Exploration in Continuous Action Spacescited by 247
- 2021Efficient and Modular Implicit Differentiationcited by 203
- 2021DC3: A learning method for optimization with hard constraintscited by 184
- 2018Applications of Deep Reinforcement Learning in Communications and Networking: A Surveycited by 167
- 2017OptLayer - Practical Constrained Optimization for Deep Reinforcement Learning in the Real Worldcited by 70
21 more
- 2017Mobile Edge Computing: A Survey on Architecture and Computation Offloadingcited by 53
- 2019Deep Reinforcement Learning meets Graph Neural Networks: exploring a routing optimization use casecited by 53
- 2019RouteNet: Leveraging Graph Neural Networks for network modeling and optimization in SDNcited by 47
- 2018Experience-driven Networking: A Deep Reinforcement Learning based Approachcited by 41
- 2021Computationally Efficient Safe Reinforcement Learning for Power Systemscited by 20
- 2021Escaping from Zero Gradient: Revisiting Action-Constrained Reinforcement Learning via Frank-Wolfe Policy Optimizationcited by 18
- 2024Modeling Other Players with Bayesian Beliefs for Games with Incomplete Informationcited by 17
- 2024FlowPG: Action-constrained Policy Gradient with Normalizing Flowscited by 16
- 2023Benchmarking Actor-Critic Deep Reinforcement Learning Algorithms for Robotics Control with Action Constraintscited by 11
- 2025Network Diffuser for Placing-Scheduling Service Function Chains with Inverse Demonstrationcited by 11
- 2026Geometry of Drifting MDPs with Path-Integral Stability Certificatescited by 8
- 2026Manifold-Constrained Energy-Based Transition Models for Offline Reinforcement Learningcited by 8
- 2021Constraint-Aware Deep Reinforcement Learning for End-to-End Resource Orchestration in Mobile Networkscited by 6
- 2023Reduced Policy Optimization for Continuous Control with Hard Constraintscited by 6
- 2026LiSFC-Search: Lifelong Search for Network SFC Optimization under Non-stationary Driftscited by 5
- 2026NonZero: Interaction-Guided Exploration for Multi-Agent Monte Carlo Tree Searchcited by 5
- 2026Operator-Guided Invariance Learning for Continuous Reinforcement Learningcited by 5
- 2026Metric-Gradient Projection for Stable Multi-Agent Policy Learningcited by 4
- 2023Safe Neural Control for Non-Affine Control Systems with Differentiable Control Barrier Functionscited by 3
- 2025SafeSlice: Enabling SLA-Compliant O-RAN Slicing via Safe Deep Reinforcement Learningcited by 1
- 2026Enabling SLO-Aware 5G Multi-Access Edge Computing with SMECcited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Wasserstein Formulation of Reinforcement Learning. An Optimal Transport Perspective on Policy Optimization
- 2026From Prior-Guided Heuristics to Deployable Agents: Accelerating Demonstration-Driven Reinforcement Learning for Deadline-Constrained Network Control
- 2024The Feasibility Theory of Constrained Reinforcement Learning: A Tutorial Study
- 2026NetRL: Powering Network Decisions via NetGRPO-Driven Reinforcement Learning
- 2026Solving Parameter-Robust Avoid Problems with Unknown Feasibility using Reinforcement Learning
- 2026Learning to Optimize: Joint Routing and Flow Allocation on Sparse Non-Euclidean Networks
- 2026Reinforcement Learning for Flow-Matching Policies with Density Transport
- 2026Constrained Flow Policy Updates: A Generalized Schr"odinger Bridge View