# Deep neural networks for choice analysis: Enhancing behavioral regularity with gradient regularization

arXiv: https://arxiv.org/abs/2404.14701 · HTML: https://mlsanity.com/paper/2404.14701/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 62 references that are in the map.
- First posted on arXiv 23 Apr 2024.

## Builds on (most cited first, 17 of 17)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Reconciling modern machine learning practice and the bias-variance trade-off](https://mlsanity.com/paper/1812.11118/) (2018)
- [Deep Visual Domain Adaptation: A Survey](https://mlsanity.com/paper/1802.03601/) (2018)
- [Towards Out-Of-Distribution Generalization: A Survey](https://mlsanity.com/paper/2108.13624/) (2021)
- [Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradients](https://mlsanity.com/paper/1711.09404/) (2017)
- [Robust Large Margin Deep Neural Networks](https://mlsanity.com/paper/1605.08254/) (2016)
- [Unconstrained Monotonic Neural Networks](https://mlsanity.com/paper/1908.05164/) (2019)
- [Improving DNN Robustness to Adversarial Attacks using Jacobian Regularization](https://mlsanity.com/paper/1803.08680/) (2018)
- [Deep Lattice Networks and Partial Monotonic Functions](https://mlsanity.com/paper/1709.06680/) (2017)
- [A Unified Gradient Regularization Family for Adversarial Examples](https://mlsanity.com/paper/1511.06385/) (2015)
- [Choice modelling in the age of machine learning -- discussion paper](https://mlsanity.com/paper/2101.11948/) (2021)
- [A Neural-embedded Choice Model: TasteNet-MNL Modeling Taste Heterogeneity with Flexibility and Interpretability](https://mlsanity.com/paper/2002.00922/) (2020)
- [How to Incorporate Monotonicity in Deep Networks While Preserving Flexibility?](https://mlsanity.com/paper/1909.10662/) (2019)
- [Theory-based residual neural networks: A synergy of discrete choice models and deep neural networks](https://mlsanity.com/paper/2010.11644/) (2020)
- [Equality of opportunity in travel behavior prediction with deep neural networks and discrete choice models](https://mlsanity.com/paper/2109.12422/) (2021)
- [Incorporating Domain Knowledge in Deep Neural Networks for Discrete Choice Models](https://mlsanity.com/paper/2306.00016/) (2023)

## Built on by (most cited first, 5 of 5)

- [Designing Graph Convolutional Neural Networks for Discrete Choice with Network Effects](https://mlsanity.com/paper/2503.09786/) (2025)
- [DIM: Enforcing Domain-Informed Monotonicity in Deep Neural Networks](https://mlsanity.com/paper/2509.21666/) (2025)
- [Bayesian Deep Learning for Discrete Choice](https://mlsanity.com/paper/2505.18077/) (2025)
- [Graph neural networks for residential location choice: connection to classical logit models](https://mlsanity.com/paper/2507.21334/) (2025)
- [Alternative Graph Neural Networks: Synergizing GEV Models and Deep Learning for Travel Mode Choice Modeling](https://mlsanity.com/paper/2509.07123/) (2025)

## Abstract (arXiv)

Deep neural networks (DNNs) frequently present behaviorally irregular patterns, significantly limiting their practical potentials and theoretical validity in travel behavior modeling. This study proposes strong and weak behavioral regularities as novel metrics to evaluate the monotonicity of individual demand functions (known as the "law of demand"), and further designs a constrained optimization framework with six gradient regularizers to enhance DNNs' behavioral regularity. The proposed framework is applied to travel survey data from Chicago and London to examine the trade-off between predictive power and behavioral regularity for large vs. small sample scenarios and in-domain vs. out-of-domain generalizations. The results demonstrate that, unlike models with strong behavioral foundations such as the multinomial logit, the benchmark DNNs cannot guarantee behavioral regularity. However, gradient regularization (GR) increases DNNs' behavioral regularity by around 6 percentage points (pp) while retaining their relatively high predictive power. In the small sample scenario, GR is more effective than in the large sample scenario, simultaneously improving behavioral regularity by about 20 pp and log-likelihood by around 1.7%. Comparing with the in-domain generalization of DNNs, GR works more effectively in out-of-domain generalization: it drastically improves the behavioral regularity of poorly performing benchmark DNNs by around 65 pp, indicating the criticality of behavioral regularization for enhancing model transferability and application in forecasting. Moreover, the proposed framework is applicable to other NN-based choice models such as TasteNets. Future studies could use behavioral regularity as a metric along with log-likelihood in evaluating travel demand models, and investigate other methods to further enhance behavioral regularity when adopting complex machine learning models.