{"list":"other","total":68,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"1512.03385","title":"Deep Residual Learning for Image Recognition","created":"2015-12-10","cited_by_count":52096},{"id":"2407.21783","title":"The Llama 3 Herd of Models","created":"2024-07-31","cited_by_count":14715},{"id":"2505.09388","title":"Qwen3 Technical Report","created":"2025-05-14","cited_by_count":8820},{"id":"1502.03167","title":"Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift","created":"2015-02-11","cited_by_count":7005},{"id":"1607.06450","title":"Layer Normalization","created":"2016-07-21","cited_by_count":5669},{"id":"2205.14135","title":"FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness","created":"2022-05-27","cited_by_count":3673},{"id":"1701.06538","title":"Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer","created":"2017-01-23","cited_by_count":3509},{"id":"1606.08415","title":"Gaussian Error Linear Units (GELUs)","created":"2016-06-27","cited_by_count":3502},{"id":"2101.03961","title":"Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity","created":"2021-01-11","cited_by_count":3073},{"id":"2408.00118","title":"Gemma 2: Improving Open Language Models at a Practical Size","created":"2024-07-31","cited_by_count":1998},{"id":"1603.05027","title":"Identity Mappings in Deep Residual Networks","created":"2016-03-16","cited_by_count":1965},{"id":"2503.19786","title":"Gemma 3 Technical Report","created":"2025-03-25","cited_by_count":1824},{"id":"2002.05202","title":"GLU Variants Improve Transformer","created":"2020-02-12","cited_by_count":1804},{"id":"1910.07467","title":"Root Mean Square Layer Normalization","created":"2019-10-16","cited_by_count":1480},{"id":"2508.10925","title":"gpt-oss-120b & gpt-oss-20b Model Card","created":"2025-08-08","cited_by_count":1386},{"id":"2002.04745","title":"On Layer Normalization in the Transformer Architecture","created":"2020-02-12","cited_by_count":922},{"id":"1312.6120","title":"Exact solutions to the nonlinear dynamics of learning in deep linear neural networks","created":"2013-12-20","cited_by_count":763},{"id":"2105.13290","title":"CogView: Mastering Text-to-Image Generation via Transformers","created":"2021-05-26","cited_by_count":694},{"id":"2507.20534","title":"Kimi K2: Open Agentic Intelligence","created":"2025-07-28","cited_by_count":567},{"id":"2103.03404","title":"Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depth","created":"2021-03-05","cited_by_count":419},{"id":"1905.13655","title":"Implicit Regularization in Deep Matrix Factorization","created":"2019-05-31","cited_by_count":405},{"id":"2505.06708","title":"Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free","created":"2025-05-10","cited_by_count":283},{"id":"2003.04887","title":"ReZero is All You Need: Fast Convergence at Large Depth","created":"2020-03-10","cited_by_count":264},{"id":"1809.10853","title":"Adaptive Input Representations for Neural Language Modeling","created":"2018-09-28","cited_by_count":226},{"id":"1911.07013","title":"Understanding and Improving Layer Normalization","created":"2019-11-16","cited_by_count":217},{"id":"1810.01075","title":"Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learning","created":"2018-10-02","cited_by_count":215},{"id":"1901.09321","title":"Fixup Initialization: Residual Learning Without Normalization","created":"2019-01-27","cited_by_count":198},{"id":"2203.00555","title":"DeepNet: Scaling Transformers to 1,000 Layers","created":"2022-03-01","cited_by_count":178},{"id":"2103.10427","title":"The Low-Rank Simplicity Bias in Deep Networks","created":"2021-03-18","cited_by_count":154},{"id":"1910.05895","title":"Transformers without Tears: Improving the Normalization of Self-Attention","created":"2019-10-14","cited_by_count":153},{"id":"2206.03126","title":"Signal Propagation in Transformers: Theoretical Perspectives and the Role of Rank Collapse","created":"2022-06-07","cited_by_count":141},{"id":"2203.05962","title":"Anti-Oversmoothing in Deep Vision Transformers via the Fourier Domain Analysis: From Theory to Practice","created":"2022-03-09","cited_by_count":136},{"id":"1711.04735","title":"Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice","created":"2017-11-13","cited_by_count":132},{"id":"2407.20311","title":"Physics of Language Models: Part 2.1, Grade-School Math and the Hidden Reasoning Process","created":"2024-07-29","cited_by_count":108},{"id":"2310.02244","title":"Tensor Programs VI: Feature Learning in Infinite-Depth Neural Networks","created":"2023-10-03","cited_by_count":97},{"id":"2110.09456","title":"NormFormer: Improved Transformer Pretraining with Extra Normalization","created":"2021-10-18","cited_by_count":83},{"id":"2505.01618","title":"Don't be lazy: CompleteP enables compute-efficient deep transformers","created":"2025-05-02","cited_by_count":78},{"id":"2202.08625","title":"Revisiting Over-smoothing in BERT from the Perspective of Graph","created":"2022-02-17","cited_by_count":72},{"id":"2309.16620","title":"Depthwise Hyperparameter Transfer in Residual Networks: Dynamics and Scaling Limit","created":"2023-09-28","cited_by_count":66},{"id":"2502.05795","title":"The Curse of Depth in Large Language Models","created":"2025-02-09","cited_by_count":62},{"id":"2010.12859","title":"Stable ResNet","created":"2020-10-24","cited_by_count":49},{"id":"2405.18781","title":"On the Role of Attention Masks and LayerNorm in Transformers","created":"2024-05-29","cited_by_count":48},{"id":"2206.06072","title":"Rank Diminishing in Deep Neural Networks","created":"2022-06-13","cited_by_count":47},{"id":"2306.17759","title":"The Shaped Transformer: Attention Models in the Infinite Depth-and-Width Limit","created":"2023-06-30","cited_by_count":47},{"id":"2302.10322","title":"Deep Transformers without Shortcuts: Modifying Self-attention for Faithful Signal Propagation","created":"2023-02-20","cited_by_count":45},{"id":"2308.01814","title":"Tensor Programs IVb: Adaptive Optimization in the Infinite-Width Limit","created":"2023-08-03","cited_by_count":38},{"id":"2311.01906","title":"Simplifying Transformer Blocks","created":"2023-11-03","cited_by_count":37},{"id":"2412.13795","title":"Mix-LN: Unleashing the Power of Deeper Layers by Combining Pre-LN and Post-LN","created":"2024-12-18","cited_by_count":37},{"id":"2502.02732","title":"Peri-LN: Revisiting Normalization Layer in the Transformer Architecture","created":"2025-02-04","cited_by_count":34},{"id":"2512.04299","title":"When do spectral gradient updates help in deep learning?","created":"2025-12-03","cited_by_count":31},{"id":"2106.03970","title":"Batch Normalization Orthogonalizes Representations in Deep Random Networks","created":"2021-06-07","cited_by_count":29},{"id":"2407.17465","title":"u-$\\mu$P: The Unit-Scaled Maximal Update Parametrization","created":"2024-07-24","cited_by_count":29},{"id":"1809.08848","title":"Dynamical Isometry is Achieved in Residual Networks in a Universal Way for any Activation Function","created":"2018-09-24","cited_by_count":22},{"id":"2401.04301","title":"Setting the Record Straight on Transformer Oversmoothing","created":"2024-01-09","cited_by_count":21},{"id":"2405.05409","title":"Initialization is Critical to Whether Transformers Fit Composite Functions by Reasoning or Memorizing","created":"2024-05-08","cited_by_count":17},{"id":"2206.00330","title":"B2T Connection: Serving Stability and Performance in Deep Transformers","created":"2022-06-01","cited_by_count":16},{"id":"2503.04598","title":"HybridNorm: Towards Stable and Efficient Transformer Training via Hybrid Normalization","created":"2025-03-06","cited_by_count":16},{"id":"2105.07741","title":"Activation function design for deep networks: linearity and effective initialisation","created":"2021-05-17","cited_by_count":13},{"id":"2602.05970","title":"Inverse Depth Scaling From Most Layers Being Similar","created":"2026-02-05","cited_by_count":9},{"id":"2505.16284","title":"Only Large Weights (And Not Skip Connections) Can Prevent the Perils of Rank Collapse","created":"2025-05-22","cited_by_count":7},{"id":"2601.19895","title":"Post-LayerNorm Is Back: Stable, ExpressivE, and Deep","created":"2026-01-27","cited_by_count":7},{"id":"2305.02790","title":"BranchNorm: Robustly Scaling Extremely Deep Transformers","created":"2023-05-04","cited_by_count":6},{"id":"2402.06751","title":"Low-Rank Learning by Design: the Role of Network Architecture and Activation Linearity in Gradient Rank Collapse","created":"2024-02-09","cited_by_count":5},{"id":"2602.18849","title":"Exact Attention Sensitivity and the Geometry of Transformer Stability","created":"2026-02-21","cited_by_count":3},{"id":"2601.20994","title":"The Depth Delusion: Why Transformers Should Be Wider, Not Deeper","created":"2026-01-28","cited_by_count":2},{"id":"2601.22580","title":"SpanNorm: Reconciling Training Stability and Performance in Deep Transformers","created":"2026-01-30","cited_by_count":2},{"id":"2510.03470","title":"On residual network depth","created":"2025-10-03","cited_by_count":1}],"next_cursor":null}