{"list":"other","total":54,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"1412.6980","title":"Adam: A Method for Stochastic Optimization","created":"2014-12-22","cited_by_count":37240},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2001.08361","title":"Scaling Laws for Neural Language Models","created":"2020-01-23","cited_by_count":6586},{"id":"1607.06450","title":"Layer Normalization","created":"2016-07-21","cited_by_count":5669},{"id":"1804.07461","title":"GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding","created":"2018-04-20","cited_by_count":4899},{"id":"1502.01852","title":"Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification","created":"2015-02-06","cited_by_count":3611},{"id":"2203.15556","title":"Training Compute-Optimal Large Language Models","created":"2022-03-29","cited_by_count":3019},{"id":"1806.07572","title":"Neural Tangent Kernel: Convergence and Generalization in Neural Networks","created":"2018-06-20","cited_by_count":2621},{"id":"2002.04745","title":"On Layer Normalization in the Transformer Architecture","created":"2020-02-12","cited_by_count":922},{"id":"2302.05442","title":"Scaling Vision Transformers to 22 Billion Parameters","created":"2023-02-10","cited_by_count":694},{"id":"1711.00165","title":"Deep Neural Networks as Gaussian Processes","created":"2017-11-01","cited_by_count":622},{"id":"1712.00409","title":"Deep Learning Scaling is Predictable, Empirically","created":"2017-12-01","cited_by_count":582},{"id":"1804.06561","title":"A Mean Field View of the Landscape of Two-Layers Neural Networks","created":"2018-04-18","cited_by_count":466},{"id":"1805.09545","title":"On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transport","created":"2018-05-24","cited_by_count":426},{"id":"2103.03404","title":"Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depth","created":"2021-03-05","cited_by_count":419},{"id":"1804.11271","title":"Gaussian Process Behaviour in Wide Deep Neural Networks","created":"2018-04-30","cited_by_count":317},{"id":"2203.03466","title":"Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer","created":"2022-03-07","cited_by_count":288},{"id":"1606.05340","title":"Exponential expressivity in deep neural networks through transient chaos","created":"2016-06-16","cited_by_count":261},{"id":"2103.09177","title":"Deep learning: a statistical viewpoint","created":"2021-03-16","cited_by_count":240},{"id":"1611.01232","title":"Deep Information Propagation","created":"2016-11-04","cited_by_count":195},{"id":"2011.14522","title":"Feature Learning in Infinite-Width Neural Networks","created":"2020-11-30","cited_by_count":190},{"id":"2106.10165","title":"The Principles of Deep Learning Theory","created":"2021-06-18","cited_by_count":173},{"id":"2206.15144","title":"Neural Networks can Learn Representations with Gradient Descent","created":"2022-06-30","cited_by_count":161},{"id":"2303.06296","title":"Stabilizing Transformer Training by Preventing Attention Entropy Collapse","created":"2023-03-11","cited_by_count":161},{"id":"2006.13409","title":"When Do Neural Networks Outperform Kernel Methods?","created":"2020-06-24","cited_by_count":150},{"id":"2205.01445","title":"High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves the Representation","created":"2022-05-03","cited_by_count":150},{"id":"2006.14548","title":"Tensor Programs II: Neural Tangent Kernel for Any Architecture","created":"2020-06-25","cited_by_count":143},{"id":"2302.11055","title":"SGD learning on neural networks: leap complexity and saddle-to-saddle dynamics","created":"2023-02-21","cited_by_count":143},{"id":"2202.08658","title":"The merged-staircase property: a necessary and nearly sufficient condition for SGD learning of sparse functions on two-layer neural networks","created":"2022-02-17","cited_by_count":142},{"id":"2206.03126","title":"Signal Propagation in Transformers: Theoretical Perspectives and the Role of Rank Collapse","created":"2022-06-07","cited_by_count":141},{"id":"2301.03728","title":"Scaling Laws for Generative Mixed-Modal Language Models","created":"2023-01-10","cited_by_count":134},{"id":"2006.10540","title":"Infinite attention: NNGP and NTK for deep attention networks","created":"2020-06-18","cited_by_count":121},{"id":"1909.05989","title":"Finite Depth and Width Corrections to the Neural Tangent Kernel","created":"2019-09-13","cited_by_count":115},{"id":"2207.10551","title":"Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling?","created":"2022-07-21","cited_by_count":103},{"id":"1902.06853","title":"On the Impact of the Activation Function on Deep Neural Networks Training","created":"2019-02-19","cited_by_count":102},{"id":"1712.08969","title":"Mean Field Residual Networks: On the Edge of Chaos","created":"2017-12-24","cited_by_count":83},{"id":"1910.00019","title":"Non-Gaussian processes and neural networks at finite widths","created":"2019-09-30","cited_by_count":78},{"id":"1909.11304","title":"Asymptotics of Wide Networks from Feynman Diagrams","created":"2019-09-25","cited_by_count":77},{"id":"2209.14863","title":"Neural Networks Efficiently Learn Low-Dimensional Representations with SGD","created":"2022-09-29","cited_by_count":63},{"id":"2303.00055","title":"Learning time-scales in two-layers neural networks","created":"2023-02-28","cited_by_count":57},{"id":"2110.01765","title":"Rapid training of deep neural networks without skip connections or normalization layers using Deep Kernel Shaping","created":"2021-10-05","cited_by_count":52},{"id":"1812.05994","title":"Products of Many Large Random Matrices and Gradients in Deep Neural Networks","created":"2018-12-14","cited_by_count":49},{"id":"2010.12859","title":"Stable ResNet","created":"2020-10-24","cited_by_count":49},{"id":"2302.10322","title":"Deep Transformers without Shortcuts: Modifying Self-attention for Faithful Signal Propagation","created":"2023-02-20","cited_by_count":45},{"id":"2206.02768","title":"The Neural Covariance SDE: Shaped Infinite Depth-and-Width Networks at Initialization","created":"2022-06-06","cited_by_count":41},{"id":"2106.04013","title":"The Future is Log-Gaussian: ResNets and Their Infinite-Depth-and-Width Limit at Initialization","created":"2021-06-07","cited_by_count":33},{"id":"2106.06615","title":"Precise characterization of the prior predictive distribution of deep ReLU networks","created":"2021-06-11","cited_by_count":28},{"id":"2203.08120","title":"Deep Learning without Shortcuts: Shaping the Kernel with Tailored Rectifiers","created":"2022-03-15","cited_by_count":28},{"id":"1809.08848","title":"Dynamical Isometry is Achieved in Residual Networks in a Universal Way for any Activation Function","created":"2018-09-24","cited_by_count":22},{"id":"2210.00688","title":"On the infinite-depth limit of finite-width neural networks","created":"2022-10-03","cited_by_count":22},{"id":"2302.00453","title":"Width and Depth Limits Commute in Residual Networks","created":"2023-02-01","cited_by_count":21},{"id":"2304.02034","title":"Effective Theory of Transformers at Initialization","created":"2023-04-04","cited_by_count":21},{"id":"2105.07741","title":"Activation function design for deep networks: linearity and effective initialisation","created":"2021-05-17","cited_by_count":13}],"next_cursor":null}