mlsanityData to 14 Oct 2026

High-Dimensional Statistics: Reflections on Progress and Open Problems

arXiv 2605.05076 ↗First posted 6 May 2026math.ST stat.CO stat.MEcited by 4

Not on the map. Why?

Abstract (arXiv)

Over the past two decades, the field of high-dimensional statistics has experienced substantial progress, driven largely by technological advances that have dramatically reduced the cost and effort for data collection and storage across a broad range of domains, including biology, medicine, astronomy, and the social and environmental sciences. Modern datasets are increasingly complex, often exhibiting rich dependency, heterogeneity, and other features that challenge traditional statistical methods. In response, high-dimensional statistics has evolved to address more sophisticated estimation and inference problems. This evolution has, in turn, fostered deep connections with and contributions to a wide range of research areas, including optimization, concentration of measure, random matrix theory, information theory, and theoretical computer science. Given the rapid pace of recent developments in high-dimensional statistics, our goal is to synthesize representative advances, highlight common themes and open problems, and point to important works that offer entry points into the field.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 81 of its 499 references that are in the map. First posted on arXiv 6 May 2026.

Builds on

81 of 499 references.

References · 81

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2020Scaling Laws for Neural Language Modelscited by 6,586
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  5. 2023Mistral 7Bcited by 3,164
  6. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  7. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  8. 2016Concrete Problems in AI Safetycited by 1,642
  9. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  10. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
  11. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 1,053
  12. 2022Toy Models of Superpositioncited by 971
38 more
  1. 2022In-context Learning and Induction Headscited by 964
  2. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
  3. 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 684
  4. 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 656
  5. 2019Certified Data Removal from Machine Learning Modelscited by 535
  6. 2023TRAK: Attributing Model Behavior at Scalecited by 283
  7. 2018The Natural Language Decathlon: Multitask Learning as Question Answeringcited by 236
  8. 2022Will we run out of data? Limits of LLM scaling based on human-generated datacited by 195
  9. 2023Localizing Model Behavior with Path Patchingcited by 195
  10. 2025Open Problems in Mechanistic Interpretabilitycited by 179
  11. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 129
  12. 2023PPI++: Efficient Prediction-Powered Inferencecited by 117
  13. 2022A Solvable Model of Neural Scaling Lawscited by 116
  14. 2019Notes on Computational Hardness of Hypothesis Testing: Predictions using the Low-Degree Likelihood Ratiocited by 85
  15. 2024Competition Dynamics Shape Algorithmic Phases of In-Context Learningcited by 54
  16. 2024Active Statistical Inferencecited by 50
  17. 2025Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifierscited by 45
  18. 2013A framework to characterize performance of LASSO algorithmscited by 40
  19. 2026Maximum Likelihood Reinforcement Learningcited by 36
  20. 2024Model Editing as a Robust and Denoised variant of DPO: A Case Study on Toxicitycited by 35
  21. 2023Assumption-Lean and Data-Adaptive Post-Prediction Inferencecited by 26
  22. 2024Asymptotics of Random Feature Regression Beyond the Linear Scaling Regimecited by 26
  23. 2013Regularly random dualitycited by 24
  24. 2025TinyV: Reducing False Negatives in Verification Improves RL for LLM Reasoningcited by 21
  25. 2023Uncovering hidden geometry in Transformers via disentangling position and contextcited by 18
  26. 2023Fully lifted random duality theorycited by 18
  27. 2025What is the objective of reasoning with reinforcement learning?cited by 18
  28. 2025Larger Datasets Can Be Repeated More: A Theoretical Analysis of Multi-Epoch Scaling in Linear Regressioncited by 16
  29. 2013Upper-bounding ℓ1-optimization weak thresholdscited by 14
  30. 2023Prediction De-Correlated Inference: A safe approach for post-prediction inferencecited by 14
  31. 2025Dynamics of Transient Structure in In-Context Linear Regression Transformerscited by 14
  32. 2025A Unified Framework for Semiparametrically Efficient Semi-Supervised Learningcited by 14
  33. 2025Asymptotics of Non-Convex Generalized Linear Models in High-Dimensions: A proof of the replica formulacited by 14
  34. 2025Improved Scaling Laws in Linear Regression via Data Reusecited by 11
  35. 2025Advantage Shaping as Surrogate Reward Maximization: Unifying Pass@K Policy Gradientscited by 11
  36. 2024Task-Agnostic Machine-Learning-Assisted Inferencecited by 10
  37. 2025When does Gaussian equivalence fail and how to fix it: Non-universal behavior of random features with quadratic scalingcited by 10
  38. 2024Local Prediction-Powered Inferencecited by 8

50 most cited of 81 All 81 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Scalable anomaly detection via a univariate Christoffel functioncited by 2
  2. 2026Lasso Universality Under Linearly Dependent Covariates in the Sparse Regimecited by 2
  3. 2026Influence Diagnostics in High-dimensional M-estimation: Precise Asymptoticscited by 0
  4. 2026Algorithms for adaptive and heteroskedastic linear regression at the computational thresholdcited by 0