{"total":53,"rows":[{"id":"2203.00555","title":"DeepNet: Scaling Transformers to 1,000 Layers","created":"2022-03-01","cited_by_count":178,"area":null},{"id":"2203.05962","title":"Anti-Oversmoothing in Deep Vision Transformers via the Fourier Domain Analysis: From Theory to Practice","created":"2022-03-09","cited_by_count":136,"area":null},{"id":"2402.16788","title":"Why Transformers Need Adam: A Hessian Perspective","created":"2024-02-26","cited_by_count":124,"area":null},{"id":"2108.12284","title":"The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformers","created":"2021-08-26","cited_by_count":117,"area":null},{"id":"2403.08540","title":"Language models scale reliably with over-training and on downstream tasks","created":"2024-03-13","cited_by_count":113,"area":null},{"id":"2406.19146","title":"Resolving Discrepancies in Compute-Optimal Scaling of Language Models","created":"2024-06-27","cited_by_count":88,"area":null},{"id":"2502.05795","title":"The Curse of Depth in Large Language Models","created":"2025-02-09","cited_by_count":62,"area":null},{"id":"2312.16903","title":"Spike No More: Stabilizing the Pre-training of Large Language Models","created":"2023-12-28","cited_by_count":52,"area":null},{"id":"2302.10322","title":"Deep Transformers without Shortcuts: Modifying Self-attention for Faithful Signal Propagation","created":"2023-02-20","cited_by_count":45,"area":null},{"id":"2311.01906","title":"Simplifying Transformer Blocks","created":"2023-11-03","cited_by_count":37,"area":null},{"id":"2203.09176","title":"ODE Transformer: An Ordinary Differential Equation-Inspired Model for Sequence Generation","created":"2022-03-17","cited_by_count":27,"area":null},{"id":"2202.00528","title":"Examining Scaling and Transfer of Language Model Architectures for Machine Translation","created":"2022-02-01","cited_by_count":26,"area":null},{"id":"2206.04571","title":"Revisiting End-to-End Speech-to-Text Translation From Scratch","created":"2022-06-09","cited_by_count":26,"area":null},{"id":"2206.10139","title":"Insights into Pre-training via Simpler Synthetic Tasks","created":"2022-06-21","cited_by_count":26,"area":null},{"id":"2501.08537","title":"Complexity Control Facilitates Reasoning-Based Compositional Generalization in Transformers","created":"2025-01-15","cited_by_count":20,"area":null},{"id":"2405.05409","title":"Initialization is Critical to Whether Transformers Fit Composite Functions by Reasoning or Memorizing","created":"2024-05-08","cited_by_count":17,"area":null},{"id":"2512.14856","title":"T5Gemma 2: Seeing, Reading, and Understanding Longer","created":"2025-12-16","cited_by_count":17,"area":null},{"id":"2206.00330","title":"B2T Connection: Serving Stability and Performance in Deep Transformers","created":"2022-06-01","cited_by_count":16,"area":null},{"id":"2503.04598","title":"HybridNorm: Towards Stable and Efficient Transformer Training via Hybrid Normalization","created":"2025-03-06","cited_by_count":16,"area":null},{"id":"2303.00579","title":"Are More Layers Beneficial to Graph Transformers?","created":"2023-03-01","cited_by_count":15,"area":null},{"id":"2104.02308","title":"ODE Transformer: An Ordinary Differential Equation-Inspired Model for Neural Machine Translation","created":"2021-04-06","cited_by_count":12,"area":null},{"id":"2206.02079","title":"Multilingual Neural Machine Translation with Deep Encoder and Multiple Shallow Decoders","created":"2022-06-05","cited_by_count":12,"area":null},{"id":"2101.00542","title":"An Efficient Transformer Decoder with Compressed Sub-layers","created":"2021-01-03","cited_by_count":10,"area":null},{"id":"2207.14467","title":"GTrans: Grouping and Fusing Transformer Layers for Neural Machine Translation","created":"2022-07-29","cited_by_count":10,"area":null},{"id":"2510.06954","title":"From Condensation to Rank Collapse: A Two-Stage Analysis of Transformer Training Dynamics","created":"2025-10-08","cited_by_count":10,"area":null},{"id":"2305.02790","title":"BranchNorm: Robustly Scaling Extremely Deep Transformers","created":"2023-05-04","cited_by_count":6,"area":null},{"id":"2405.18194","title":"Delving into Differentially Private Transformer","created":"2024-05-28","cited_by_count":5,"area":null},{"id":"2410.05052","title":"Initialization of Large Language Models via Reparameterization to Mitigate Loss Spikes","created":"2024-10-07","cited_by_count":4,"area":null},{"id":"2503.17500","title":"Variance Control via Weight Rescaling in LLM Pre-training","created":"2025-03-21","cited_by_count":4,"area":null},{"id":"2603.15389","title":"When Does Sparsity Mitigate the Curse of Depth in LLMs","created":"2026-03-16","cited_by_count":4,"area":null},{"id":"2108.10417","title":"Recurrent multiple shared layers in Depth for Neural Machine Translation","created":"2021-08-23","cited_by_count":3,"area":null},{"id":"2403.07921","title":"Merino: Entropy-driven Design for Generative Language Models on IoT Devices","created":"2024-02-28","cited_by_count":3,"area":null},{"id":"2502.15499","title":"Scale-Distribution Decoupling: Enabling Stable and Effective Training of Large Language Models","created":"2025-02-21","cited_by_count":3,"area":null},{"id":"2211.16174","title":"CUNI Submission in WMT22 General Task","created":"2022-11-29","cited_by_count":2,"area":null},{"id":"2305.17633","title":"DPFormer: Learning Differentially Private Transformer on Long-Tailed Data","created":"2023-05-28","cited_by_count":2,"area":null},{"id":"2405.13901","title":"Discrete Cosine Transform Based Decorrelated Attention for Vision Transformers","created":"2024-05-22","cited_by_count":2,"area":null},{"id":"2408.05738","title":"Language-Informed Beam Search Decoding for Multilingual Machine Translation","created":"2024-08-11","cited_by_count":2,"area":null},{"id":"2410.15578","title":"Generalized Probabilistic Attention Mechanism in Transformers","created":"2024-10-21","cited_by_count":2,"area":null},{"id":"2109.08008","title":"The NiuTrans System for WNGT 2020 Efficiency Task","created":"2021-09-16","cited_by_count":1,"area":null},{"id":"2109.09354","title":"CUNI systems for WMT21: Multilingual Low-Resource Translation for Indo-European Languages Shared Task","created":"2021-09-20","cited_by_count":1,"area":null},{"id":"2302.10871","title":"Efficient CTC Regularization via Coarse Labels for End-to-End Speech Translation","created":"2023-02-21","cited_by_count":1,"area":null},{"id":"2601.22095","title":"GeoNorm: Unify Pre-Norm and Post-Norm with Geodesic Optimization","created":"2026-01-29","cited_by_count":1,"area":null},{"id":"2603.05369","title":"Progressive Residual Warmup for Language Model Pretraining","created":"2026-03-05","cited_by_count":1,"area":null},{"id":"2608.12218","title":"Information Abundance Paradox: Long-Context Training Undermines Parametric Knowledge","created":"2026-08-12","cited_by_count":1,"area":null},{"id":"2108.03340","title":"Tiny Neural Models for Seq2Seq","created":"2021-08-07","cited_by_count":0,"area":null},{"id":"2212.12662","title":"Optimizing Deep Transformers for Chinese-Thai Low-Resource Translation","created":"2022-12-24","cited_by_count":0,"area":null},{"id":"2306.09338","title":"Understanding Optimization of Deep Learning via Jacobian Matrix and Lipschitz Constant","created":"2023-06-15","cited_by_count":0,"area":null},{"id":"2404.11870","title":"Enhancing Length Extrapolation in Sequential Models with Pointer-Augmented Neural Memory","created":"2024-04-18","cited_by_count":0,"area":null},{"id":"2407.19044","title":"Advancing Neural Network Performance through Emergence-Promoting Initialization Scheme","created":"2024-07-26","cited_by_count":0,"area":null},{"id":"2606.07527","title":"Post-training is (Massive) Supervised Learning","created":"2026-04-20","cited_by_count":0,"area":null},{"id":"2606.16112","title":"Scaling Adaptive Depth with Norm-Agnostic Residual Networks","created":"2026-06-15","cited_by_count":0,"area":null},{"id":"2608.09417","title":"Why Post-Norm Transformers Collapse: Attention Amplification and Gradient Repair Failure","created":"2026-08-10","cited_by_count":0,"area":null},{"id":"2608.14689","title":"A Reproducibility Study of Partial Residual Ablations in Pre-LN Transformers","created":"2026-08-06","cited_by_count":0,"area":null}],"next_cursor":null}