# Deep hybrid model with satellite imagery: how to combine demand modeling and computer vision for behavior analysis?

arXiv: https://arxiv.org/abs/2303.04204 · HTML: https://mlsanity.com/paper/2303.04204/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 96 references that are in the map.
- First posted on arXiv 7 Mar 2023.

## Builds on (most cited first, 30 of 33)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [Autoencoding beyond pixels using a learned similarity metric](https://mlsanity.com/paper/1512.09300/) (2015)
- [Adversarial Feature Learning](https://mlsanity.com/paper/1605.09782/) (2016)
- [A Neural Representation of Sketch Drawings](https://mlsanity.com/paper/1704.03477/) (2017)
- [Adversarially Learned Inference](https://mlsanity.com/paper/1606.00704/) (2016)
- [The Benefit of Multitask Representation Learning](https://mlsanity.com/paper/1505.06279/) (2015)
- [Human Trajectory Forecasting in Crowds: A Deep Learning Perspective](https://mlsanity.com/paper/2007.03639/) (2020)
- [Inductive Graph Neural Networks for Spatiotemporal Kriging](https://mlsanity.com/paper/2006.07527/) (2020)
- [Understanding and Improving Interpolation in Autoencoders via an Adversarial Regularizer](https://mlsanity.com/paper/1807.07543/) (2018)
- [Using Deep Learning and Google Street View to Estimate the Demographic Makeup of the US](https://mlsanity.com/paper/1702.06683/) (2017)
- [Deep Learning the City : Quantifying Urban Perception At A Global Scale](https://mlsanity.com/paper/1608.01769/) (2016)
- [Multi-level Convolutional Autoencoder Networks for Parametric Prediction of Spatio-temporal Dynamics](https://mlsanity.com/paper/1912.11114/) (2019)
- [Uncertainty Quantification of Sparse Travel Demand Prediction with Spatial-Temporal Graph Neural Networks](https://mlsanity.com/paper/2208.05908/) (2022)
- [Generating Interpretable Poverty Maps using Object Detection in Satellite Images](https://mlsanity.com/paper/2002.01612/) (2020)
- [Spatial Aggregation and Temporal Convolution Networks for Real-time Kriging](https://mlsanity.com/paper/2109.12144/) (2021)
- [Choice modelling in the age of machine learning -- discussion paper](https://mlsanity.com/paper/2101.11948/) (2021)
- [Using convolutional networks and satellite imagery to identify patterns in urban environments at a large scale](https://mlsanity.com/paper/1704.02965/) (2017)
- [Theory-based residual neural networks: A synergy of discrete choice models and deep neural networks](https://mlsanity.com/paper/2010.11644/) (2020)
- [Adversarially Approximated Autoencoder for Image Generation and Manipulation](https://mlsanity.com/paper/1902.05581/) (2019)
- [Combining Discrete Choice Models and Neural Networks through Embeddings: Formulation, Interpretability and Performance](https://mlsanity.com/paper/2109.12042/) (2021)
- [Equality of opportunity in travel behavior prediction with deep neural networks and discrete choice models](https://mlsanity.com/paper/2109.12422/) (2021)
- [Autoencoder Regularized Network For Driving Style Representation Learning](https://mlsanity.com/paper/1701.01272/) (2017)
- [Ensemble Convolutional Neural Networks for Mode Inference in Smartphone Travel Survey](https://mlsanity.com/paper/1904.08933/) (2019)
- [Semi-supervised GANs to Infer Travel Modes in GPS Trajectories](https://mlsanity.com/paper/1902.10768/) (2019)

## Built on by (most cited first, 8 of 8)

- [Aligning LLM with human travel choices: a persona-based embedding learning approach](https://mlsanity.com/paper/2505.19003/) (2025)
- [Vision Technologies with Applications in Traffic Surveillance Systems: A Holistic Survey](https://mlsanity.com/paper/2412.00348/) (2024)
- [Mitigating Spatial Disparity in Urban Prediction Using Residual-Aware Spatiotemporal Graph Neural Networks: A Chicago Case Study](https://mlsanity.com/paper/2501.11214/) (2025)
- [Images in Discrete Choice Modeling: Addressing Data Isomorphism in Multi-Modality Inputs](https://mlsanity.com/paper/2312.14724/) (2023)
- [Demographics-Informed Neural Network for Multi-Modal Spatiotemporal forecasting of Urban Growth and Travel Patterns Using Satellite Imagery](https://mlsanity.com/paper/2506.12456/) (2025)
- [Advancing Transportation Mode Share Analysis with Built Environment: Deep Hybrid Models with Urban Road Network](https://mlsanity.com/paper/2405.14079/) (2024)
- [A Unified Framework for Next-Gen Urban Forecasting via LLM-driven Dependency Retrieval and GeoTransformer](https://mlsanity.com/paper/2408.08852/) (2024)
- [Learning Multimodal Embeddings for Traffic Accident Prediction and Causal Estimation](https://mlsanity.com/paper/2512.02920/) (2025)

## Abstract (arXiv)

Classical demand modeling analyzes travel behavior using only low-dimensional numeric data (i.e. sociodemographics and travel attributes) but not high-dimensional urban imagery. However, travel behavior depends on the factors represented by both numeric data and urban imagery, thus necessitating a synergetic framework to combine them. This study creates a theoretical framework of deep hybrid models with a crossing structure consisting of a mixing operator and a behavioral predictor, thus integrating the numeric and imagery data into a latent space. Empirically, this framework is applied to analyze travel mode choice using the MyDailyTravel Survey from Chicago as the numeric inputs and the satellite images as the imagery inputs. We found that deep hybrid models outperform both the traditional demand models and the recent deep learning in predicting the aggregate and disaggregate travel behavior with our supervision-as-mixing design. The latent space in deep hybrid models can be interpreted, because it reveals meaningful spatial and social patterns. The deep hybrid models can also generate new urban images that do not exist in reality and interpret them with economic theory, such as computing substitution patterns and social welfare changes. Overall, the deep hybrid models demonstrate the complementarity between the low-dimensional numeric and high-dimensional imagery data and between the traditional demand modeling and recent deep learning. It generalizes the latent classes and variables in classical hybrid demand models to a latent space, and leverages the computational power of deep learning for imagery while retaining the economic interpretability on the microeconomics foundation.