# Expert Level control of Ramp Metering based on Multi-task Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/1701.08832 · HTML: https://mlsanity.com/paper/1701.08832/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jan 2017.

## Built on by (most cited first, 16 of 16)

- [Unified Automatic Control of Vehicular Systems with Reinforcement Learning](https://mlsanity.com/paper/2208.00268/) (2022)
- [Temporal Transfer Learning for Traffic Optimization with Coarse-grained Advisory Autonomy](https://mlsanity.com/paper/2312.09436/) (2023)
- [SHAIL: Safety-Aware Hierarchical Adversarial Imitation Learning for Autonomous Driving in Urban Environments](https://mlsanity.com/paper/2204.01922/) (2022)
- [Building Safer Autonomous Agents by Leveraging Risky Driving Behavior Knowledge](https://mlsanity.com/paper/2103.10245/) (2021)
- [Reinforcement Learning with Model Predictive Control for Highway Ramp Metering](https://mlsanity.com/paper/2311.08820/) (2023)
- [Robust Dynamic Bus Control: A Distributional Multi-agent Reinforcement Learning Approach](https://mlsanity.com/paper/2111.01946/) (2021)
- [A Bibliometric Analysis and Review on Reinforcement Learning for Transportation Applications](https://mlsanity.com/paper/2210.14524/) (2022)
- [Towards Integrated Traffic Control with Operating Decentralized Autonomous Organization](https://mlsanity.com/paper/2308.03769/) (2023)
- [Demonstration-guided Deep Reinforcement Learning for Coordinated Ramp Metering and Perimeter Control in Large Scale Networks](https://mlsanity.com/paper/2303.03395/) (2023)
- [Reinforcement Learning for Freeway Lane-Change Regulation via Connected Vehicles](https://mlsanity.com/paper/2412.04341/) (2024)
- [PMA-Diffusion: A Physics-guided Mask-Aware Diffusion Framework for TSE from Sparse Observations](https://mlsanity.com/paper/2512.06183/) (2025)
- [Sharing the Control Authority Between Deep Reinforcement Learning and Model Predictive Control: Application to Multi-Class Transportation Networks](https://mlsanity.com/paper/2608.20858/) (2026)
- [Analyzing Robustness of the Deep Reinforcement Learning Algorithm in Ramp Metering Applications Considering False Data Injection Attack and Defense](https://mlsanity.com/paper/2301.12036/) (2023)
- [TrafficClaw: A Generalizable LLM Agent in the Unified Physical Environment for Urban Traffic Control](https://mlsanity.com/paper/2604.17456/) (2026)
- [LLM-Based Hierarchical Coordinated Control with Continuation-Aware Policy Learning](https://mlsanity.com/paper/2608.15041/) (2026)
- [Learning to Move Cities: Deep Meta-Models and Reinforcement Policies for Calibration and Control in Urban Networks](https://mlsanity.com/paper/2609.21945/) (2026)

## Abstract (arXiv)

This article shows how the recent breakthroughs in Reinforcement Learning (RL) that have enabled robots to learn to play arcade video games, walk or assemble colored bricks, can be used to perform other tasks that are currently at the core of engineering cyberphysical systems. We present the first use of RL for the control of systems modeled by discretized non-linear Partial Differential Equations (PDEs) and devise a novel algorithm to use non-parametric control techniques for large multi-agent systems. We show how neural network based RL enables the control of discretized PDEs whose parameters are unknown, random, and time-varying. We introduce an algorithm of Mutual Weight Regularization (MWR) which alleviates the curse of dimensionality of multi-agent control schemes by sharing experience between agents while giving each agent the opportunity to specialize its action policy so as to tailor it to the local parameters of the part of the system it is located in.