mlsanityData to 7 Oct 2026

DNA-SE: Towards Deep Neural-Nets Assisted Semiparametric Estimation

arXiv 2408.02045 ↗First posted 4 Aug 2024stat.ML cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Semiparametric statistics play a pivotal role in a wide range of domains, including but not limited to missing data, causal inference, and transfer learning, to name a few. In many settings, semiparametric theory leads to (nearly) statistically optimal procedures that yet involve numerically solving Fredholm integral equations of the second kind. Traditional numerical methods, such as polynomial or spline approximations, are difficult to scale to multi-dimensional problems. Alternatively, statisticians may choose to approximate the original integral equations by ones with closed-form solutions, resulting in computationally more efficient, but statistically suboptimal or even incorrect procedures. To bridge this gap, we propose a novel framework by formulating the semiparametric estimation problem as a bi-level optimization problem; and then we develop a scalable algorithm called Deep Neural-Nets Assisted Semiparametric Estimation (DNA-SE) by leveraging the universal approximation property of Deep Neural-Nets (DNN) to streamline semiparametric procedures. Through extensive numerical experiments and a real data analysis, we demonstrate the numerical and statistical advantages of dnase over traditional methods. To the best of our knowledge, we are the first to bring DNN into semiparametric statistics as a numerical solver of integral equations in our proposed general framework.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 37 of its 77 references that are in the map. First posted on arXiv 4 Aug 2024.

Builds on

37 of 77 references.

References · 37

  1. 2017Attention Is All You Needcited by 59,932
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  3. 2017The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problemscited by 581
  4. 2017Nonparametric regression using deep neural networks with ReLU activation functioncited by 431
  5. 2020Generative Language Modeling for Automated Theorem Provingcited by 379
  6. 2017The Marginal Value of Adaptive Gradient Methods in Machine Learningcited by 248
  7. 2018Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionalitycited by 160
  8. 2020Towards Theoretically Understanding Why SGD Generalizes Better Than ADAM in Deep Learningcited by 125
  9. 2017Implicit Regularization in Deep Learningcited by 90
  10. 2018How Well Generative Adversarial Networks Learn Distributionscited by 78
  11. 2020Semiparametric proximal causal inferencecited by 58
  12. 2020Superpolynomial Lower Bounds for Learning One-Layer Neural Networks using Gradient Descentcited by 57
25 more
  1. 2017Tensor Networks for Dimensionality Reduction and Large-Scale Optimizations. Part 2 Applications and Future Perspectivescited by 53
  2. 2021Machine Learning For Elliptic PDEs: Fast Rate Generalization Bound, Neural Scaling Law and Minimax Optimalitycited by 40
  3. 2022Nonparametric Masked Language Modelingcited by 39
  4. 2020Domain Adaptation as a Problem of Inference on Graphical Modelscited by 36
  5. 2023Subspace Identification for Multi-Source Domain Adaptationcited by 36
  6. 2020Quantile regression with deep ReLU Networks: Estimators and minimax ratescited by 24
  7. 2022Knowledge-in-Context: Towards Knowledgeable Semi-Parametric Language Modelscited by 24
  8. 2019A consensus-based global optimization method for high dimensional machine learning problemscited by 23
  9. 2018A Generalized Neyman-Pearson Criterion for Optimal Domain Adaptationcited by 22
  10. 2020Double Generative Adversarial Networks for Conditional Independence Testingcited by 21
  11. 2018Gradient Descent for One-Hidden-Layer Neural Networks: Polynomial Convergence and SQ Lower Boundscited by 18
  12. 2022Neural Integro-Differential Equationscited by 17
  13. 2022Deep Learning Methods for Proximal Inference via Maximum Moment Restrictioncited by 13
  14. 2022DeepMed: Semiparametric Causal Mediation Analysis with Debiased Deep Learningcited by 10
  15. 2024Simplifying debiased inference via automatic differentiation and probabilistic programmingcited by 10
  16. 2023ELSA: Efficient Label Shift Adaptation through the Lens of Semiparametric Modelscited by 9
  17. 2023Ananke: A Python Package For Causal Inference Using Graphical Modelscited by 8
  18. 2020Causal Inference of General Treatment Effects using Neural Networks with A Diverging Number of Confounderscited by 7
  19. 2020Learning sums of powers of low-degree polynomials in the non-degenerate casecited by 6
  20. 2019Efficient estimation of optimal regimes under a no direct effect assumptioncited by 5
  21. 2021Efficient Estimation in NPIV Models: A Comparison of Various Neural Networks-Based Estimatorscited by 5
  22. 2023Efficient and Multiply Robust Risk Estimation under General Forms of Dataset Shiftcited by 5
  23. 2019A Versatile Estimation Procedure without Estimating the Nonignorable Missingness Mechanismcited by 4
  24. 2022Solving Fredholm Integral Equations of the First Kind via Wasserstein Gradient Flowscited by 4
  25. 2019A Semiparametric Approach to Model-based Sensitivity Analysis in Observational Studiescited by 2

Built on by

Not cited yet.