# Deep Reinforcement Learning for Day-to-day Dynamic Tolling in Tradable Credit Schemes

arXiv: https://arxiv.org/abs/2504.08074 · HTML: https://mlsanity.com/paper/2504.08074/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods.
- Builds on 12 of its 55 references that are in the map.
- First posted on arXiv 10 Apr 2025.

## Placed in

- Deep reinforcement learning methods / [Adversarial robustness and RL in finance](https://mlsanity.com/deeprl/adversarial-robustness-and-rl-in-finance/)

## Builds on (most cited first, 12 of 12)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Regularizing Action Policies for Smooth Control with Reinforcement Learning](https://mlsanity.com/paper/2012.06644/) (2020)
- [L2C2: Locally Lipschitz Continuous Constraint towards Stable and Smooth Reinforcement Learning](https://mlsanity.com/paper/2202.07152/) (2022)
- [TAAC: Temporally Abstract Actor-Critic for Continuous Control](https://mlsanity.com/paper/2104.06521/) (2021)
- [Addressing Action Oscillations through Learning Policy Inertia](https://mlsanity.com/paper/2103.02287/) (2021)
- [Deep Reinforcement Learning Algorithm for Dynamic Pricing of Express Lanes with Multiple Access Locations](https://mlsanity.com/paper/1909.04760/) (2019)
- [Flight Controller Synthesis Via Deep Reinforcement Learning](https://mlsanity.com/paper/1909.06493/) (2019)
- [Managing network congestion with a tradable credit scheme: a trip-based MFD approach](https://mlsanity.com/paper/2009.06965/) (2020)
- [Bayesian Optimization of Area-based Road Pricing](https://mlsanity.com/paper/2012.11047/) (2020)
- [Real-Time Predictive Control Strategy Optimization](https://mlsanity.com/paper/1901.04571/) (2019)

## Abstract (arXiv)

Tradable credit schemes (TCS) are an increasingly studied alternative to congestion pricing, given their revenue neutrality and ability to address issues of equity through the initial credit allocation. Modeling TCS to aid future design and implementation is associated with challenges involving user and market behaviors, demand-supply dynamics, and control mechanisms. In this paper, we focus on the latter and address the day-to-day dynamic tolling problem under TCS, which is formulated as a discrete-time Markov Decision Process and solved using reinforcement learning (RL) algorithms. Our results indicate that RL algorithms achieve travel times and social welfare comparable to the Bayesian optimization benchmark, with generalization across varying capacities and demand levels. We further assess the robustness of RL under different hyperparameters and apply regularization techniques to mitigate action oscillation, which generates practical tolling strategies that are transferable under day-to-day demand and supply variability. Finally, we discuss potential challenges such as scaling to large networks, and show how transfer learning can be leveraged to improve computational efficiency and facilitate the practical deployment of RL-based TCS solutions.