{"list":"other","total":54,"rows":[{"id":"1512.03385","title":"Deep Residual Learning for Image Recognition","created":"2015-12-10","cited_by_count":52096},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2010.11929","title":"An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale","created":"2020-10-22","cited_by_count":30186},{"id":"1912.01703","title":"PyTorch: An Imperative Style, High-Performance Deep Learning Library","created":"2019-12-03","cited_by_count":20305},{"id":"2302.13971","title":"LLaMA: Open and Efficient Foundation Language Models","created":"2023-02-27","cited_by_count":14132},{"id":"2001.08361","title":"Scaling Laws for Neural Language Models","created":"2020-01-23","cited_by_count":6586},{"id":"2101.03961","title":"Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity","created":"2021-01-11","cited_by_count":3073},{"id":"2203.15556","title":"Training Compute-Optimal Large Language Models","created":"2022-03-29","cited_by_count":3019},{"id":"1806.07572","title":"Neural Tangent Kernel: Convergence and Generalization in Neural Networks","created":"2018-06-20","cited_by_count":2621},{"id":"2106.04560","title":"Scaling Vision Transformers","created":"2021-06-08","cited_by_count":999},{"id":"2002.04745","title":"On Layer Normalization in the Transformer Architecture","created":"2020-02-12","cited_by_count":922},{"id":"1902.06720","title":"Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent","created":"2019-02-18","cited_by_count":738},{"id":"1812.07956","title":"On Lazy Training in Differentiable Programming","created":"2018-12-19","cited_by_count":675},{"id":"1711.00165","title":"Deep Neural Networks as Gaussian Processes","created":"2017-11-01","cited_by_count":622},{"id":"1805.09545","title":"On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transport","created":"2018-05-24","cited_by_count":426},{"id":"1804.11271","title":"Gaussian Process Behaviour in Wide Deep Neural Networks","created":"2018-04-30","cited_by_count":317},{"id":"2203.03466","title":"Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer","created":"2022-03-07","cited_by_count":288},{"id":"2002.04486","title":"Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Loss","created":"2020-02-11","cited_by_count":268},{"id":"2003.04887","title":"ReZero is All You Need: Fast Convergence at Large Depth","created":"2020-03-10","cited_by_count":264},{"id":"1901.09321","title":"Fixup Initialization: Residual Learning Without Normalization","created":"2019-01-27","cited_by_count":198},{"id":"1902.06015","title":"Mean-field theory of two-layers neural networks: dimension-free bounds and kernel limit","created":"2019-02-16","cited_by_count":187},{"id":"1702.08591","title":"The Shattered Gradients Problem: If resnets are the answer, then what is the question?","created":"2017-02-28","cited_by_count":157},{"id":"2006.14548","title":"Tensor Programs II: Neural Tangent Kernel for Any Architecture","created":"2020-06-25","cited_by_count":143},{"id":"2206.03126","title":"Signal Propagation in Transformers: Theoretical Perspectives and the Role of Rank Collapse","created":"2022-06-07","cited_by_count":141},{"id":"1909.05989","title":"Finite Depth and Width Corrections to the Neural Tangent Kernel","created":"2019-09-13","cited_by_count":115},{"id":"2205.09653","title":"Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networks","created":"2022-05-19","cited_by_count":105},{"id":"1801.03744","title":"Which Neural Net Architectures Give Rise To Exploding and Vanishing Gradients?","created":"2018-01-11","cited_by_count":94},{"id":"1910.00019","title":"Non-Gaussian processes and neural networks at finite widths","created":"2019-09-30","cited_by_count":78},{"id":"2012.04030","title":"Statistical Mechanics of Deep Linear Neural Networks: The Back-Propagating Kernel Renormalization","created":"2020-12-07","cited_by_count":71},{"id":"2006.06098","title":"Dynamical mean-field theory for stochastic gradient descent in Gaussian mixture classification","created":"2020-06-10","cited_by_count":60},{"id":"1812.05994","title":"Products of Many Large Random Matrices and Gradients in Deep Neural Networks","created":"2018-12-14","cited_by_count":49},{"id":"2010.12859","title":"Stable ResNet","created":"2020-10-24","cited_by_count":49},{"id":"2306.17759","title":"The Shaped Transformer: Attention Models in the Infinite Depth-and-Width Limit","created":"2023-06-30","cited_by_count":47},{"id":"2002.03432","title":"On the distance between two neural networks and the stability of learning","created":"2020-02-09","cited_by_count":46},{"id":"2304.03408","title":"Dynamics of Finite Width Kernel and Prediction Fluctuations in Mean Field Neural Networks","created":"2023-04-06","cited_by_count":43},{"id":"2206.02768","title":"The Neural Covariance SDE: Shaped Infinite Depth-and-Width Networks at Initialization","created":"2022-06-06","cited_by_count":41},{"id":"2305.18411","title":"Feature-Learning Networks Are Consistent Across Widths At Realistic Scales","created":"2023-05-28","cited_by_count":41},{"id":"2106.04013","title":"The Future is Log-Gaussian: ResNets and Their Infinite-Depth-and-Width Limit at Initialization","created":"2021-06-07","cited_by_count":33},{"id":"2106.03970","title":"Batch Normalization Orthogonalizes Representations in Deep Random Networks","created":"2021-06-07","cited_by_count":29},{"id":"2106.06615","title":"Precise characterization of the prior predictive distribution of deep ReLU networks","created":"2021-06-11","cited_by_count":28},{"id":"2303.17671","title":"Neural signature kernels as infinite-width-depth-limits of controlled ResNets","created":"2023-03-30","cited_by_count":25},{"id":"1809.08848","title":"Dynamical Isometry is Achieved in Residual Networks in a Universal Way for any Activation Function","created":"2018-09-24","cited_by_count":22},{"id":"1908.10030","title":"Finite size corrections for neural network Gaussian processes","created":"2019-08-27","cited_by_count":22},{"id":"2210.00688","title":"On the infinite-depth limit of finite-width neural networks","created":"2022-10-03","cited_by_count":22},{"id":"2212.14457","title":"Bayesian Interpolation with Deep Linear Networks","created":"2022-12-29","cited_by_count":21},{"id":"2302.00453","title":"Width and Depth Limits Commute in Residual Networks","created":"2023-02-01","cited_by_count":21},{"id":"2112.05589","title":"Unified field theoretical approach to deep and recurrent neuronal networks","created":"2021-12-10","cited_by_count":20},{"id":"2210.02157","title":"The Influence of Learning Rule on Representation Dynamics in Wide Neural Networks","created":"2022-10-05","cited_by_count":19},{"id":"2209.13435","title":"Scaling Laws For Deep Learning Based Image Reconstruction","created":"2022-09-27","cited_by_count":14},{"id":"2304.05187","title":"Automatic Gradient Descent: Deep Learning without Hyperparameters","created":"2023-04-11","cited_by_count":14},{"id":"2305.18399","title":"On the impact of activation and normalization in obtaining isometric embeddings at initialization","created":"2023-05-28","cited_by_count":13},{"id":"1709.02956","title":"Deep Residual Networks and Weight Initialization","created":"2017-09-09","cited_by_count":12},{"id":"2305.07810","title":"Depth Dependence of $\\mu$P Learning Rates in ReLU MLPs","created":"2023-05-13","cited_by_count":9},{"id":"2305.07715","title":"Field theory for optimal signal propagation in ResNets","created":"2023-05-12","cited_by_count":6}],"next_cursor":null}