# Differentiable Policy Transport over Multi-Layer Network Feasibility Geometry

arXiv: https://arxiv.org/abs/2609.26068 · HTML: https://mlsanity.com/paper/2609.26068/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 55 references that are in the map.
- First posted on arXiv 2 Aug 2026.

## Builds on (most cited first, 30 of 33)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [OptNet: Differentiable Optimization as a Layer in Neural Networks](https://mlsanity.com/paper/1703.00443/) (2017)
- [Differentiable Convex Optimization Layers](https://mlsanity.com/paper/1910.12430/) (2019)
- [Reward Constrained Policy Optimization](https://mlsanity.com/paper/1805.11074/) (2018)
- [A Lyapunov-based Approach to Safe Reinforcement Learning](https://mlsanity.com/paper/1805.07708/) (2018)
- [Safe Exploration in Continuous Action Spaces](https://mlsanity.com/paper/1801.08757/) (2018)
- [Efficient and Modular Implicit Differentiation](https://mlsanity.com/paper/2105.15183/) (2021)
- [DC3: A learning method for optimization with hard constraints](https://mlsanity.com/paper/2104.12225/) (2021)
- [Applications of Deep Reinforcement Learning in Communications and Networking: A Survey](https://mlsanity.com/paper/1810.07862/) (2018)
- [OptLayer - Practical Constrained Optimization for Deep Reinforcement Learning in the Real World](https://mlsanity.com/paper/1709.07643/) (2017)
- [Mobile Edge Computing: A Survey on Architecture and Computation Offloading](https://mlsanity.com/paper/1702.05309/) (2017)
- [Deep Reinforcement Learning meets Graph Neural Networks: exploring a routing optimization use case](https://mlsanity.com/paper/1910.07421/) (2019)
- [RouteNet: Leveraging Graph Neural Networks for network modeling and optimization in SDN](https://mlsanity.com/paper/1910.01508/) (2019)
- [Experience-driven Networking: A Deep Reinforcement Learning based Approach](https://mlsanity.com/paper/1801.05757/) (2018)
- [Computationally Efficient Safe Reinforcement Learning for Power Systems](https://mlsanity.com/paper/2110.10333/) (2021)
- [Escaping from Zero Gradient: Revisiting Action-Constrained Reinforcement Learning via Frank-Wolfe Policy Optimization](https://mlsanity.com/paper/2102.11055/) (2021)
- [Modeling Other Players with Bayesian Beliefs for Games with Incomplete Information](https://mlsanity.com/paper/2405.14122/) (2024)
- [FlowPG: Action-constrained Policy Gradient with Normalizing Flows](https://mlsanity.com/paper/2402.05149/) (2024)
- [Benchmarking Actor-Critic Deep Reinforcement Learning Algorithms for Robotics Control with Action Constraints](https://mlsanity.com/paper/2304.08743/) (2023)
- [Network Diffuser for Placing-Scheduling Service Function Chains with Inverse Demonstration](https://mlsanity.com/paper/2501.05673/) (2025)
- [Geometry of Drifting MDPs with Path-Integral Stability Certificates](https://mlsanity.com/paper/2601.21991/) (2026)
- [Manifold-Constrained Energy-Based Transition Models for Offline Reinforcement Learning](https://mlsanity.com/paper/2602.02900/) (2026)
- [Constraint-Aware Deep Reinforcement Learning for End-to-End Resource Orchestration in Mobile Networks](https://mlsanity.com/paper/2110.04320/) (2021)
- [Reduced Policy Optimization for Continuous Control with Hard Constraints](https://mlsanity.com/paper/2310.09574/) (2023)
- [LiSFC-Search: Lifelong Search for Network SFC Optimization under Non-stationary Drifts](https://mlsanity.com/paper/2602.14360/) (2026)
- [NonZero: Interaction-Guided Exploration for Multi-Agent Monte Carlo Tree Search](https://mlsanity.com/paper/2605.00751/) (2026)
- [Operator-Guided Invariance Learning for Continuous Reinforcement Learning](https://mlsanity.com/paper/2605.06500/) (2026)
- [Metric-Gradient Projection for Stable Multi-Agent Policy Learning](https://mlsanity.com/paper/2605.18809/) (2026)

## Built on by (most cited first, 1 of 1)

- [Geometry-Aware Operator Families for Structured Representation Learning](https://mlsanity.com/paper/2609.33089/) (2026)

## Abstract (arXiv)

Learning-based control is increasingly central to automating network operations. A learned policy, however, must satisfy cross-layer constraints on interference, power-rate coupling, flow conservation, service chains, capacity, latency, and reliability. Existing methods typically account for only a subset of this geometry and only indirectly, e.g., through reward penalties, Lagrange multipliers, or post-hoc repairs. This paper proposes Network Feasibility Geometry Reinforcement Learning (NFG-RL), which models coupled constraints via transport theory and the residual inclusion bphi_mathfrakN(x,a)∈cK_mathfrakN, defining the executed policy as the pushforward of a proto-policy through a feasibility-transport map. NFG-RL compiles heterogeneous constraints into typed residual blocks and transports proto-actions through a differentiable variational operator, letting active constraints shape execution, exploration, and actor gradients. Our analysis shows that exact transport yields almost-sure feasible execution, while active constraints contract exploration onto the feasible tangent space. It further establishes a nonnegative first-order gain from critic-tilted transport over plain projection and recovers backpressure scheduling as the gradient of a lifted drift residual. In two public-trace-conditioned wireless-edge surrogate environments, NFG-RL improves feasible utility by 37.5--41.5% over the strongest non-NFG method in each environment, reduces raw-action violation by 48.5--60.8%, and lowers P99 delay by 57.0--75.5%, outperforming a range of optimization and learning baselines.