# Predicting Census Survey Response Rates With Parsimonious Additive Models and Structured Interactions

arXiv: https://arxiv.org/abs/2108.11328 · HTML: https://mlsanity.com/paper/2108.11328/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 77 references that are in the map.
- First posted on arXiv 24 Aug 2021.

## Builds on (most cited first, 14 of 14)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [InterpretML: A Unified Framework for Machine Learning Interpretability](https://mlsanity.com/paper/1909.09223/) (2019)
- [Coordinate Descent Algorithms](https://mlsanity.com/paper/1502.04759/) (2015)
- [Best Subset Selection via a Modern Optimization Lens](https://mlsanity.com/paper/1507.03133/) (2015)
- [GAMI-Net: An Explainable Neural Network based on Generalized Additive Models with Structured Interactions](https://mlsanity.com/paper/2003.07132/) (2020)
- [Fast Best Subset Selection: Coordinate Descent and Local Combinatorial Optimization Algorithms](https://mlsanity.com/paper/1803.01454/) (2018)
- [Sparse Regression at Scale: Branch-and-Bound rooted in First-Order Optimization](https://mlsanity.com/paper/2004.06152/) (2020)
- [Subset Selection with Shrinkage: Sparse Linear Modeling when the SNR is low](https://mlsanity.com/paper/1708.03288/) (2017)
- [Minimax Optimal Rates of Estimation in High Dimensional Additive Models: Universal Phase Transition](https://mlsanity.com/paper/1503.02817/) (2015)
- [Grouped Variable Selection with Discrete Optimization: Computational and Statistical Perspectives](https://mlsanity.com/paper/2104.07084/) (2021)
- [Hierarchical Sparse Modeling: A Choice of Two Group Lasso Formulations](https://mlsanity.com/paper/1512.01631/) (2015)
- [The Discrete Dantzig Selector: Estimating Sparse Linear Models via Mixed Integer Linear Optimization](https://mlsanity.com/paper/1508.01922/) (2015)
- [Learning Hierarchical Interactions at Scale: A Convex Optimization Approach](https://mlsanity.com/paper/1902.01542/) (2019)

## Built on by (most cited first, 3 of 3)

- [End-to-end Feature Selection Approach for Learning Skinny Trees](https://mlsanity.com/paper/2310.18542/) (2023)
- [Neural Additive Experts: Context-Gated Experts for Controllable Model Additivity](https://mlsanity.com/paper/2602.10585/) (2026)
- [ParamBoost: Gradient Boosted Piecewise Cubic Polynomials](https://mlsanity.com/paper/2604.18864/) (2026)

## Abstract (arXiv)

In this paper, we consider the problem of predicting survey response rates using a family of flexible and interpretable nonparametric models. The study is motivated by the US Census Bureau's well-known ROAM application, which uses a linear regression model trained on the US Census Planning Database data to identify hard-to-survey areas. A crowdsourcing competition (Erdman and Bates, 2016) organized more than ten years ago revealed that machine learning methods based on ensembles of regression trees led to the best performance in predicting survey response rates; however, the corresponding models could not be adopted for the intended application due to their black-box nature. We consider nonparametric additive models with a small number of main and pairwise interaction effects using ℓ0-based penalization. From a methodological viewpoint, we study our estimator's computational and statistical aspects and discuss variants incorporating strong hierarchical interactions. Our algorithms (open-sourced on GitHub) extend the computational frontiers of existing algorithms for sparse additive models to be able to handle datasets relevant to the application we consider. We discuss and interpret findings from our model on the US Census Planning Database. In addition to being useful from an interpretability standpoint, our models lead to predictions comparable to popular black-box machine learning methods based on gradient boosting and feedforward neural networks - suggesting that it is possible to have models that have the best of both worlds: good model accuracy and interpretability.