mlsanityData to 7 Oct 2026

Benchmarking the Neural Linear Model for Regression

arXiv 1912.08416 ↗First posted 18 Dec 2019stat.ML cs.LGcited by 31

Not on the map. Why?

Abstract (arXiv)

The neural linear model is a simple adaptive Bayesian linear regression method that has recently been used in a number of problems ranging from Bayesian optimization to reinforcement learning. Despite its apparent successes in these settings, to the best of our knowledge there has been no systematic exploration of its capabilities on simple regression tasks. In this work we characterize these on the UCI datasets, a popular benchmark for Bayesian regression models, as well as on the recently introduced UCI "gap" datasets, which are better tests of out-of-distribution uncertainty. We demonstrate that the neural linear model is a simple method that shows generally good performance on these tasks, but at the cost of requiring good hyperparameter tuning.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Dec 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 4; 2023: 8; 2024: 6; 2025: 2; 2026: 3.

31 papers.

Sort:

By area:

  1. 2021The Promises and Pitfalls of Deep Kernel Learningcited by 81
  2. 2024Variational Bayesian Last Layerscited by 67
  3. 2022Do Bayesian Neural Networks Need To Be Fully Stochastic?cited by 63
  4. 2022A Framework and Benchmark for Deep Batch Active Learning for Regressioncited by 47
  5. 2021Mixtures of Laplace Approximations for Improved Post-Hoc Uncertainty in Deep Learningcited by 25
  6. 2021Bayesian Deep Basis Fitting for Depth Completion with Uncertaintycited by 23
  7. 2021NOMU: Neural Optimization-based Model Uncertaintycited by 16
  8. 2024Bayesian Optimization via Continual Variational Last Layer Trainingcited by 13
  9. 2021Trust Your Robots! Predictive Uncertainty Estimation of Neural Networks with Sparse Gaussian Processescited by 12
  10. 2023A Symmetry-Aware Exploration of Bayesian Neural Network Posteriorscited by 12
  11. 2023Improved uncertainty quantification for neural networks with Bayesian last layercited by 11
  12. 2022Bayesian Optimization-based Combinatorial Assignmentcited by 9
19 more
  1. 2023Incorporating Unlabelled Data into Bayesian Neural Networkscited by 9
  2. 2023Scalable Stochastic Gradient Riemannian Langevin Dynamics in Non-Diagonal Metricscited by 8
  3. 2022Towards a Unified Framework for Uncertainty-aware Nonlinear Variable Selection with Theoretical Guaranteescited by 4
  4. 2023Density Uncertainty Layers for Reliable Uncertainty Estimationcited by 4
  5. 2023On Feynman--Kac training of partial Bayesian neural networkscited by 4
  6. 2021Uncertainty Quantification in Neural Differential Equationscited by 3
  7. 2026Spectral methods: crucial for machine learning, natural for quantum computers?cited by 3
  8. 2021Marginally calibrated response distributions for end-to-end learning in autonomous drivingcited by 2
  9. 2024Achieving Well-Informed Decision-Making in Drug Discovery: A Comprehensive Calibration Study using Neural Network-Based Structure-Activity Modelscited by 2
  10. 2025Enhancing Uncertainty Estimation and Interpretability via Bayesian Non-negative Decision Layercited by 2
  11. 2023Probabilistic Unrolling: Scalable, Inverse-Free Maximum Likelihood Estimation for Latent Gaussian Modelscited by 1
  12. 2025Approximating Full Conformal Prediction for Neural Network Regression with Gauss-Newton Influencecited by 1
  13. 2021Understanding Uncertainty in Bayesian Deep Learningcited by 0
  14. 2023FineMorphs: Affine-diffeomorphic sequences for regressioncited by 0
  15. 2024Multivariate Bayesian Last Layer for Regression with Uncertainty Quantification and Decompositioncited by 0
  16. 2024Structured Partial Stochasticity in Bayesian Neural Networkscited by 0
  17. 2024Stein Variational Newton Neural Network Ensemblescited by 0
  18. 2026Beliefs Beyond Posteriors: The Predictive Cost of ELBO Bindingcited by 0
  19. 2026Joint Model and Data Sparsification via the Marginal Likelihoodcited by 0