{"list":"other","total":79,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"1512.03385","title":"Deep Residual Learning for Image Recognition","created":"2015-12-10","cited_by_count":52096},{"id":"1710.09412","title":"mixup: Beyond Empirical Risk Minimization","created":"2017-10-25","cited_by_count":5183},{"id":"1905.00414","title":"Similarity of Neural Network Representations Revisited","created":"2019-05-01","cited_by_count":1755},{"id":"1611.03530","title":"Understanding deep learning requires rethinking generalization","created":"2016-11-10","cited_by_count":1674},{"id":"1706.02677","title":"Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour","created":"2017-06-08","cited_by_count":1492},{"id":"2010.01412","title":"Sharpness-Aware Minimization for Efficiently Improving Generalization","created":"2020-10-03","cited_by_count":1417},{"id":"1609.04836","title":"On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima","created":"2016-09-15","cited_by_count":1352},{"id":"1803.05407","title":"Averaging Weights Leads to Wider Optima and Better Generalization","created":"2018-03-14","cited_by_count":1209},{"id":"1712.09913","title":"Visualizing the Loss Landscape of Neural Nets","created":"2017-12-28","cited_by_count":1130},{"id":"1812.11118","title":"Reconciling modern machine learning practice and the bias-variance trade-off","created":"2018-12-28","cited_by_count":927},{"id":"1805.12152","title":"Robustness May Be at Odds with Accuracy","created":"2018-05-30","cited_by_count":880},{"id":"1912.02292","title":"Deep Double Descent: Where Bigger Models and More Data Hurt","created":"2019-12-04","cited_by_count":672},{"id":"1912.05671","title":"Linear Mode Connectivity and the Lottery Ticket Hypothesis","created":"2019-12-11","cited_by_count":637},{"id":"1802.10026","title":"Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs","created":"2018-02-27","cited_by_count":588},{"id":"1811.03804","title":"Gradient Descent Finds Global Minima of Deep Neural Networks","created":"2018-11-09","cited_by_count":574},{"id":"1906.11300","title":"Benign Overfitting in Linear Regression","created":"2019-06-26","cited_by_count":549},{"id":"1703.11008","title":"Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data","created":"2017-03-31","cited_by_count":533},{"id":"1912.02178","title":"Fantastic Generalization Measures and Where to Find Them","created":"2019-12-04","cited_by_count":513},{"id":"1703.04933","title":"Sharp Minima Can Generalize For Deep Nets","created":"2017-03-15","cited_by_count":466},{"id":"1912.02757","title":"Deep Ensembles: A Loss Landscape Perspective","created":"2019-12-05","cited_by_count":443},{"id":"1908.05355","title":"The generalization error of random features regression: Precise asymptotics and double descent curve","created":"2019-08-14","cited_by_count":413},{"id":"1905.03696","title":"HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precision","created":"2019-04-29","cited_by_count":402},{"id":"1611.01838","title":"Entropy-SGD: Biasing Gradient Descent Into Wide Valleys","created":"2016-11-06","cited_by_count":359},{"id":"1909.05840","title":"Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT","created":"2019-09-12","cited_by_count":359},{"id":"1805.11604","title":"How Does Batch Normalization Help Optimization?","created":"2018-05-29","cited_by_count":357},{"id":"1803.00885","title":"Essentially No Barriers in Neural Network Energy Landscape","created":"2018-03-02","cited_by_count":330},{"id":"1707.09564","title":"A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks","created":"2017-07-29","cited_by_count":314},{"id":"1912.07145","title":"PyHessian: Neural Networks Through the Lens of the Hessian","created":"2019-12-16","cited_by_count":292},{"id":"1406.2572","title":"Identifying and attacking the saddle point problem in high-dimensional non-convex optimization","created":"2014-06-10","cited_by_count":289},{"id":"1703.00887","title":"How to Escape Saddle Points Efficiently","created":"2017-03-02","cited_by_count":285},{"id":"1412.0233","title":"The Loss Surfaces of Multilayer Networks","created":"2014-11-30","cited_by_count":280},{"id":"1503.02101","title":"Escaping From Saddle Points --- Online Stochastic Gradient for Tensor Decomposition","created":"2015-03-06","cited_by_count":243},{"id":"1812.04754","title":"Gradient Descent Happens in a Tiny Subspace","created":"2018-12-12","cited_by_count":217},{"id":"1810.01075","title":"Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learning","created":"2018-10-02","cited_by_count":215},{"id":"1901.06053","title":"A Tail-Index Analysis of Stochastic Gradient Noise in Deep Neural Networks","created":"2019-01-18","cited_by_count":214},{"id":"1903.07571","title":"Two models of double descent for weak features","created":"2019-03-18","cited_by_count":195},{"id":"1802.08760","title":"Sensitivity and Generalization in Neural Networks: an Empirical Study","created":"2018-02-23","cited_by_count":189},{"id":"1412.6544","title":"Qualitatively characterizing neural network optimization problems","created":"2014-12-19","cited_by_count":188},{"id":"2010.15110","title":"Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the Neural Tangent Kernel","created":"2020-10-28","cited_by_count":187},{"id":"1512.05287","title":"A Theoretically Grounded Application of Dropout in Recurrent Neural Networks","created":"2015-12-16","cited_by_count":161},{"id":"1611.07476","title":"Eigenvalues of the Hessian in Deep Learning: Singularity and Beyond","created":"2016-11-22","cited_by_count":159},{"id":"1705.09886","title":"Convergence Analysis of Two-layer Neural Networks with ReLU Activation","created":"2017-05-28","cited_by_count":137},{"id":"1707.04926","title":"Theoretical insights into the optimization landscape of over-parameterized shallow neural networks","created":"2017-07-16","cited_by_count":128},{"id":"1903.09139","title":"Harmless interpolation of noisy data in regression","created":"2019-03-21","cited_by_count":121},{"id":"1802.06175","title":"An Alternative View: When Does SGD Escape Local Minima?","created":"2018-02-17","cited_by_count":115},{"id":"2002.06716","title":"Predicting trends in the quality of state-of-the-art neural networks without access to training or testing data","created":"2020-02-17","cited_by_count":115},{"id":"2002.11328","title":"Rethinking Bias-Variance Trade-off for Generalization of Neural Networks","created":"2020-02-26","cited_by_count":107},{"id":"1901.08276","title":"Traditional and Heavy-Tailed Self Regularization in Neural Network Models","created":"2019-01-24","cited_by_count":105},{"id":"1712.08968","title":"Spurious Local Minima are Common in Two-Layer ReLU Neural Networks","created":"2017-12-24","cited_by_count":104},{"id":"1611.01540","title":"Topology and Geometry of Half-Rectified Network Optimization","created":"2016-11-04","cited_by_count":102},{"id":"2008.06786","title":"The Neural Tangent Kernel in High Dimensions: Triple Descent and a Multi-Scale Theory of Generalization","created":"2020-08-15","cited_by_count":101},{"id":"2008.11865","title":"Traces of Class/Cross-Class Structure Pervade Deep Learning Spectra","created":"2020-08-27","cited_by_count":94},{"id":"1902.00744","title":"Asymmetric Valleys: Beyond Sharp and Flat Local Minima","created":"2019-02-02","cited_by_count":90},{"id":"1802.08241","title":"Hessian-based Analysis of Large Batch Training and Robustness to Adversaries","created":"2018-02-22","cited_by_count":87},{"id":"1802.08770","title":"A Walk with SGD","created":"2018-02-24","cited_by_count":78},{"id":"1705.10412","title":"Gradient Descent Can Take Exponential Time to Escape Saddle Points","created":"2017-05-29","cited_by_count":74},{"id":"2006.05013","title":"A Random Matrix Analysis of Random Fourier Features: Beyond the Gaussian Kernel, a Precise Phase Transition, and the Corresponding Double Descent","created":"2020-06-09","cited_by_count":70},{"id":"1906.04724","title":"Large Scale Structure of Neural Network Loss Landscapes","created":"2019-06-11","cited_by_count":68},{"id":"1810.08591","title":"A Modern Take on the Bias-Variance Tradeoff in Neural Networks","created":"2018-10-19","cited_by_count":65},{"id":"2006.03509","title":"Triple descent and the two kinds of overfitting: Where & why do they appear?","created":"2020-06-05","cited_by_count":65},{"id":"1906.06247","title":"Explaining Landscape Connectivity of Low-cost Solutions for Multilayer Nets","created":"2019-06-14","cited_by_count":63},{"id":"1809.09349","title":"The jamming transition as a paradigm to understand the loss landscape of deep neural networks","created":"2018-09-25","cited_by_count":62},{"id":"2007.01429","title":"The Global Landscape of Neural Networks: An Overview","created":"2020-07-02","cited_by_count":57},{"id":"1901.07417","title":"On Connected Sublevel Sets in Deep Learning","created":"2019-01-22","cited_by_count":51},{"id":"1910.05929","title":"Emergent properties of the local geometry of neural loss landscapes","created":"2019-10-14","cited_by_count":46},{"id":"2008.01036","title":"Multiple Descent: Design Your Own Generalization Curve","created":"2020-08-03","cited_by_count":45},{"id":"1901.08278","title":"Heavy-Tailed Universality Predicts Trends in Test Accuracies for Very Large Pre-Trained Deep Neural Networks","created":"2019-01-24","cited_by_count":44},{"id":"2001.07384","title":"Understanding Why Neural Networks Generalize Well Through GSNR of Parameters","created":"2020-01-21","cited_by_count":40},{"id":"1804.10200","title":"The loss landscape of overparameterized neural networks","created":"2018-04-26","cited_by_count":39},{"id":"1912.04533","title":"Exact expressions for double descent and implicit regularization via surrogate random design","created":"2019-12-10","cited_by_count":35},{"id":"1912.10095","title":"Landscape Connectivity and Dropout Stability of SGD Solutions for Over-parameterized Neural Networks","created":"2019-12-20","cited_by_count":35},{"id":"1707.09920","title":"Regularization techniques for fine-tuning in neural machine translation","created":"2017-07-31","cited_by_count":27},{"id":"1710.09553","title":"Rethinking generalization requires revisiting old ideas: statistical mechanics approaches and complex learning behavior","created":"2017-10-26","cited_by_count":25},{"id":"2006.09091","title":"Flatness is a False Friend","created":"2020-06-16","cited_by_count":19},{"id":"2106.00734","title":"Post-mortem on a deep learning contest: a Simpson's paradox and the complementary roles of scale metrics versus shape metrics","created":"2021-06-01","cited_by_count":19},{"id":"1810.01021","title":"Large batch size training of neural networks with adversarial training and second-order information","created":"2018-10-02","cited_by_count":18},{"id":"1301.7115","title":"Statistical mechanics of complex neural systems and high dimensional data","created":"2013-01-30","cited_by_count":15},{"id":"1608.06671","title":"Emergence of low noise \\emph{frustrated} states in E/I balanced neural networks","created":"2016-08-23","cited_by_count":1}],"next_cursor":null}