mlsanityData to 14 Oct 2026

deep-significance - Easy and Meaningful Statistical Significance Testing in the Age of Neural Networks

arXiv 2204.06815 ↗First posted 14 Apr 2022cs.LGcited by 28

Not on the map. Why?

Abstract (arXiv)

A lot of Machine Learning (ML) and Deep Learning (DL) research is of an empirical nature. Nevertheless, statistical significance testing (SST) is still not widely used. This endangers true progress, as seeming improvements over a baseline might be statistical flukes, leading follow-up research astray while wasting human and computational resources. Here, we provide an easy-to-use package containing different significance tests and utility functions specifically tailored towards research needs and usability.

Key facts. Cited by 28 papers in the map (data through 14 Oct 2026). Builds on 22 of its 54 references that are in the map. First posted on arXiv 14 Apr 2022.

Builds on

22 of 54 references.

References · 22

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2016OpenAI Gymcited by 1,736
  4. 2017Visualizing the Loss Landscape of Neural Netscited by 1,130
  5. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  6. 2017Deep Reinforcement Learning that Matterscited by 754
  7. 2021Accounting for Variance in Machine Learning Benchmarkscited by 201
  8. 2018Deep Reinforcement Learning and the Deadly Triadcited by 174
  9. 2019Show Your Work: Improved Reporting of Experimental Resultscited by 169
  10. 2018Model Evaluation, Model Selection, and Algorithm Selection in Machine Learningcited by 157
  11. 2022Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Textcited by 155
  12. 2020With Little Power Comes Great Responsibilitycited by 142
10 more
  1. 2021The Benchmark Lotterycited by 136
  2. 2020Descending through a Crowded Valley - Benchmarking Deep Learning Optimizerscited by 119
  3. 2016Time for a change: a tutorial for comparing multiple classifiers through Bayesian analysiscited by 93
  4. 2021Do Transformer Modifications Transfer Across Implementations and Applications?cited by 87
  5. 2021Scientific Credibility of Machine Translation Research: A Meta-Evaluation of 769 Paperscited by 71
  6. 2017An optimal transportation approach for assessing almost stochastic ordercited by 33
  7. 2021Hyperparameter Optimization Is Deceiving Us, and How to Stop Itcited by 20
  8. 2018Why Comparing Single Performance Scores Does Not Allow to Draw Conclusions About Machine Learning Approachescited by 15
  9. 2019Not All Claims are Created Equal: Choosing the Right Statistical Approach to Assess Hypothesescited by 8
  10. 2016Models for the assessment of treatment improvement: the ideal and the feasiblecited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 10; 2024: 7; 2025: 4; 2026: 4.

28 papers.

Sort:

By area:

  1. 2022LingMess: Linguistically Informed Multi Expert Scorers for Coreference Resolutioncited by 26
  2. 2022Exploring Predictive Uncertainty and Calibration in NLP: A Study on the Impact of Method & Data Scarcitycited by 25
  3. 2024A noisy elephant in the room: Is your out-of-distribution detector robust to label noise?cited by 13
  4. 2023On Retrospective k-space Subsampling schemes For Deep MRI Reconstructioncited by 12
  5. 2023VideoAdviser: Video Knowledge Distillation for Multimodal Transfer Learningcited by 6
  6. 2022DynDepNet: Learning Time-Varying Dependency Structures from fMRI Data via Dynamic Graph Structure Learningcited by 5
  7. 2023Context-aware attention layers coupled with optimal transport domain adaptation and multimodal fusion methods for recognizing dementia from spontaneous speechcited by 5
  8. 2023Tokenization with Factorized Subword Encodingcited by 4
  9. 2023Class Uncertainty: A Measure to Mitigate Class Imbalancecited by 4
  10. 2025Generalized Attention Flow: Feature Attribution for Transformer Models via Maximum Flowcited by 4
  11. 2023Risk Aware Benchmarking of Large Language Modelscited by 2
  12. 2023Collaboration or Corporate Capture? Quantifying NLP's Reliance on Industry Artifacts and Contributionscited by 2
16 more
  1. 2024End-to-end Adaptive Dynamic Subsampling and Reconstruction for Cardiac MRIcited by 2
  2. 2024Multivariate Stochastic Dominance via Optimal Transport and Applications to Models Benchmarkingcited by 2
  3. 2026pAI/MSc: ML Theory Research with Humans on the Loopcited by 2
  4. 2026Topics as Proxies for Sociodemographics: How Conversational Context Affects LLM Answerscited by 2
  5. 2023DBGDGM: Dynamic Brain Graph Deep Generative Modelcited by 1
  6. 2023Triggering Dark Showers with Conditional Dual Auto-Encoderscited by 1
  7. 2024NeuralSolver: Learning Algorithms For Consistent and Efficient Extrapolation Across General Taskscited by 1
  8. 2024A multimodal LLM for the non-invasive decoding of spoken text from brain recordingscited by 1
  9. 2026Tail-Shape Estimation in LLM Evaluation Is Fragile: A Protocol for Diagnosing False Positivescited by 1
  10. 2023Estimating Motor Symptom Presence and Severity in Parkinson's Disease from Wrist Accelerometer Time Series using ROCKET and InceptionTimecited by 0
  11. 2024A Target-Aware Analysis of Data Augmentation for Hate Speech Detectioncited by 0
  12. 2024Enriching Social Science Research via Survey Item Linkingcited by 0
  13. 2025Modelling Adjectival Modification Effects on Semantic Plausibilitycited by 0
  14. 2025CorefInst: Leveraging LLMs for Multilingual Coreference Resolutioncited by 0
  15. 2025ManufactuBERT: Efficient Continual Pretraining for Manufacturingcited by 0
  16. 2026DEEP: Docker-based Execution and Evaluation Platformcited by 0