# A deep real options policy for sequential service region design and timing

arXiv: https://arxiv.org/abs/2212.14800 · HTML: https://mlsanity.com/paper/2212.14800/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 114 references that are in the map.
- First posted on arXiv 30 Dec 2022.

## Builds on (most cited first, 10 of 10)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [LSTM: A Search Space Odyssey](https://mlsanity.com/paper/1503.04069/) (2015)
- [Recurrent Neural Networks for Time Series Forecasting: Current Status and Future Directions](https://mlsanity.com/paper/1909.00590/) (2019)
- [From Reinforcement Learning to Optimal Control: A unified framework for sequential decisions](https://mlsanity.com/paper/1912.03513/) (2019)
- [Shuffling Recurrent Neural Networks](https://mlsanity.com/paper/2007.07324/) (2020)
- [Worldwide city transport typology prediction with sentence-BERT based supervised learning via Wikipedia](https://mlsanity.com/paper/2204.05193/) (2022)
- [Dynamic UAV-based traffic monitoring under uncertainty as a stochastic arc-inventory routing policy](https://mlsanity.com/paper/1609.03201/) (2016)
- [A Framework to Integrate Mode Choice in the Design of Mobility-on-Demand Systems](https://mlsanity.com/paper/1805.06094/) (2018)

## Abstract (arXiv)

As various city agencies and mobility operators navigate toward innovative mobility solutions, there is a need for strategic flexibility in well-timed investment decisions in the design and timing of mobility service regions, i.e. cast as "real options" (RO). This problem becomes increasingly challenging with multiple interacting RO in such investments. We propose a scalable machine learning based RO framework for multi-period sequential service region design & timing problem for mobility-on-demand services, framed as a Markov decision process with non-stationary stochastic variables. A value function approximation policy from literature uses multi-option least squares Monte Carlo simulation to get a policy value for a set of interdependent investment decisions as deferral options (CR policy). The goal is to determine the optimal selection and timing of a set of zones to include in a service region. However, prior work required explicit enumeration of all possible sequences of investments. To address the combinatorial complexity of such enumeration, we propose a new variant "deep" RO policy using an efficient recurrent neural network (RNN) based ML method (CR-RNN policy) to sample sequences to forego the need for enumeration, making network design & timing policy tractable for large scale implementation. Experiments on multiple service region scenarios in New York City (NYC) shows the proposed policy substantially reduces the overall computational cost (time reduction for RO evaluation of > 90% of total investment sequences is achieved), with zero to near-zero gap compared to the benchmark. A case study of sequential service region design for expansion of MoD services in Brooklyn, NYC show that using the CR-RNN policy to determine optimal RO investment strategy yields a similar performance (0.5% within CR policy value) with significantly reduced computation time (about 5.4 times faster).