# Consideration Circuits: Depth Separation and Universality Beyond a Single Softmax

arXiv: https://arxiv.org/abs/2610.04143 · HTML: https://mlsanity.com/paper/2610.04143/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 43 references that are in the map.
- First posted on arXiv 2 Oct 2026.

## Builds on (most cited first, 11 of 11)

- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [Benefits of depth in neural networks](https://mlsanity.com/paper/1602.04485/) (2016)
- [The Power of Depth for Feedforward Neural Networks](https://mlsanity.com/paper/1512.03965/) (2015)
- [A Function Space View of Bounded Norm Infinite Width ReLU Nets: The Multivariate Case](https://mlsanity.com/paper/1910.01635/) (2019)
- [A Neural-embedded Choice Model: TasteNet-MNL Modeling Taste Heterogeneity with Flexibility and Interpretability](https://mlsanity.com/paper/2002.00922/) (2020)
- [Discovering Context Effects from Raw Choice Data](https://mlsanity.com/paper/1902.03266/) (2019)
- [Representing Random Utility Choice Models with Neural Networks](https://mlsanity.com/paper/2207.12877/) (2022)
- [Pairwise Choice Markov Chains](https://mlsanity.com/paper/1603.02740/) (2016)
- [Transformer Choice Net: A Transformer Neural Network for Choice Prediction](https://mlsanity.com/paper/2310.08716/) (2023)
- [Depth Separation in Norm-Bounded Infinite-Width Neural Networks](https://mlsanity.com/paper/2402.08808/) (2024)
- [DeepHalo: A Neural Choice Model with Controllable Context Effects](https://mlsanity.com/paper/2601.04616/) (2026)

## Abstract (arXiv)

Most feature-based choice models, classical and deep, score items and apply a single softmax. We introduce consideration circuits (CC), feature-based models of multi-stage choice defined by directed acyclic graphs of multinomial logit (MNL) units. Source units assign probabilities to menu items, and internal units combine predecessor distributions using MNL weights computed from their probability-weighted feature summaries. On a three-item compromise task with fixed non-collinear features, menu-independent random-utility models (RUM), including a single MNL unit, suffer an error bounded away from zero. For CC, in contrast, we establish a sharp depth--norm separation: increasing depth from 2 to 3 reduces the optimal maximum taste-vector norm for error ε from Theta(log(1/ε)/ε) to Theta(log(1/ε)). The depth-2 lower bound holds for arbitrary width and menu-independent routing biases, while a five-node depth-3 circuit with zero routing biases attains the logarithmic rate. More generally, we characterize two geometric conditions that are necessary and sufficient for approximating arbitrary deterministic choice tables on finite menu families. Under these conditions, depth 3 suffices, while depth 4 achieves optimal logarithmic norm scaling whenever the family contains a non-singleton menu. In experiments, standalone tree circuits with fewer than 600 parameters attain the lowest mean test negative log-likelihood (NLL) among the evaluated models on four fixed-pool benchmarks and the Expedia temporal split. As output heads, CC generalize the linear MNL readout and lower mean test NLL for every tested encoder on Expedia and Trivago.