# Share, Collaborate, Benchmark: Advancing Travel Demand Research through rigorous open-source collaboration

arXiv: https://arxiv.org/abs/2306.06194 · HTML: https://mlsanity.com/paper/2306.06194/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 62 references that are in the map.
- First posted on arXiv 9 Jun 2023.

## Builds on (most cited first, 6 of 6)

- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)](https://mlsanity.com/paper/2003.12206/) (2020)
- [Conditional Time Series Forecasting with Convolutional Neural Networks](https://mlsanity.com/paper/1703.04691/) (2017)
- [sktime: A Unified Interface for Machine Learning with Time Series](https://mlsanity.com/paper/1909.07872/) (2019)
- [The Use of Machine Learning Algorithms in Recommender Systems: A Systematic Review](https://mlsanity.com/paper/1511.05263/) (2015)
- [Multi-output Bus Travel Time Prediction with Convolutional LSTM Neural Network](https://mlsanity.com/paper/1903.02791/) (2019)

## Abstract (arXiv)

This research foregrounds general practices in travel demand research, emphasizing the need to change our ways. A critical barrier preventing travel demand literature from effectively informing policy is the volume of publications without clear, consolidated benchmarks, making it difficult for researchers and policymakers to gather insights and use models to guide decision-making. By emphasizing reproducibility and open collaboration, we aim to enhance the reliability and policy relevance of travel demand research. We present a collaborative infrastructure for transit demand prediction models, focusing on their performance during highly dynamic conditions like the COVID-19 pandemic. Drawing from over 300 published papers, we develop an open-source infrastructure with five common methodologies and assess their performance under stable and dynamic conditions. We found that the prediction error for the LSTM deep learning approach stabilized at a mean arctangent absolute percentage error (MAAPE) of about 0.12 within 1.5 months, whereas other models continued to exhibit higher error rates even a year into the pandemic. If research practices had prioritized reproducibility before the COVID-19 pandemic, transit agencies would have had clearer guidance on the best forecasting methods and quickly identified those best suited for pandemic conditions to inform operations in response to changes in transit demand. The aim of this open-source codebase is to lower the barrier for other researchers to replicate, reproduce models and build upon findings. We encourage researchers to test their own modeling approaches on this benchmarking platform, challenge the analyses conducted in this paper, and develop model specifications that can outperform those evaluated here. Further, collaborative research approaches must be expanded across travel demand modeling if we wish to impact policy and planning.