mlsanityData to 14 Oct 2026

What can we Learn by Predicting Accuracy?

arXiv 2208.01358 ↗First posted 2 Aug 2022cs.LG cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

This paper seeks to answer the following question: textit"What can we learn by predicting accuracy?". Indeed, classification is one of the most popular tasks in machine learning, and many loss functions have been developed to maximize this non-differentiable objective function. Unlike past work on loss function design, which was guided mainly by intuition and theory before being validated by experimentation, here we propose to approach this problem in the opposite way: we seek to extract knowledge by experimentation. This data-driven approach is similar to that used in physics to discover general laws from data. We used a symbolic regression method to automatically find a mathematical expression highly correlated with a linear classifier's accuracy. The formula discovered on more than 260 datasets of embeddings has a Pearson's correlation of 0.96 and a r2 of 0.93. More interestingly, this formula is highly explainable and confirms insights from various previous papers on loss design. We hope this work will open new perspectives in the search for new heuristics leading to a deeper understanding of machine learning theory.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 33 of its 64 references that are in the map. First posted on arXiv 2 Aug 2022.

Builds on

33 of 64 references.

References · 33

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2016Densely Connected Convolutional Networkscited by 6,720
  4. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  5. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
  6. 2013Describing Textures in the Wildcited by 2,125
  7. 2019Searching for MobileNetV3cited by 1,994
  8. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,850
  9. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,060
  10. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,057
  11. 2020Meta-Learning in Neural Networks: A Surveycited by 950
  12. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 742
21 more
  1. 2019AI Feynman: a Physics-Inspired Method for Symbolic Regressioncited by 548
  2. 2019Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradientscited by 309
  3. 2020Whitening for Self-Supervised Representation Learningcited by 279
  4. 2021Contemporary Symbolic Regression Methods and their Relative Performancecited by 215
  5. 2019Scaling and Benchmarking Self-Supervised Visual Representation Learningcited by 201
  6. 2019FSS-1000: A 1000-Class Dataset for Few-Shot Segmentationcited by 160
  7. 2021On Feature Decorrelation in Self-Supervised Learningcited by 160
  8. 2019Neural Predictor for Neural Architecture Searchcited by 118
  9. 2020Are Labels Always Necessary for Classifier Accuracy Evaluation?cited by 104
  10. 2021Symbolic Regression via Neural-Guided Genetic Programming Population Seedingcited by 101
  11. 2015Optimal whitening and decorrelationcited by 97
  12. 2022Symbolic Regression is NP-hardcited by 71
  13. 2018How Complex is your classification problem? A survey on measuring classification complexitycited by 66
  14. 2018Can We Gain More from Orthogonality Regularizations in Training Deep CNNs?cited by 62
  15. 2018Rethinking Feature Distribution for Loss Functions in Image Classificationcited by 39
  16. 2015Deep Linear Discriminant Analysiscited by 23
  17. 2020Fisher Discriminant Triplet and Contrastive Losses for Training Siamese Networkscited by 15
  18. 2018Efficient Image Dataset Classification Difficulty Estimation for Predicting Deep-Learning Accuracycited by 14
  19. 2021Symbolic regression outperforms other models for small data setscited by 12
  20. 2018TAPAS: Train-less Accuracy Predictor for Architecture Searchcited by 11
  21. 2018Evolutionary Data Measures: Understanding the Difficulty of Text Classification Taskscited by 11

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 2.

3 papers.

  1. 2023LOVM: Language-Only Vision Model Selectioncited by 12
  2. 2023Unsupervised Evaluation of Out-of-distribution Detection: A Data-centric Perspectivecited by 4
  3. 2022Fine-tune your Classifier: Finding Correlations With Temperaturecited by 0