{"total":198,"rows":[{"id":"2103.17239","title":"Going deeper with Image Transformers","created":"2021-03-31","cited_by_count":764,"area":null},{"id":"2102.06171","title":"High-Performance Large-Scale Image Recognition Without Normalization","created":"2021-02-11","cited_by_count":360,"area":null},{"id":"2204.13650","title":"Unlocking High-Accuracy Differentially Private Image Classification through Scale","created":"2022-04-28","cited_by_count":224,"area":null},{"id":"2302.11552","title":"Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMC","created":"2023-02-22","cited_by_count":220,"area":"diffusion"},{"id":"2112.00059","title":"Evaluating Gradient Inversion Attacks and Defenses in Federated Learning","created":"2021-11-30","cited_by_count":196,"area":null},{"id":"2310.08659","title":"LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Models","created":"2023-10-12","cited_by_count":196,"area":null},{"id":"2312.03863","title":"Efficient Large Language Models: A Survey","created":"2023-12-06","cited_by_count":186,"area":null},{"id":"2203.00555","title":"DeepNet: Scaling Transformers to 1,000 Layers","created":"2022-03-01","cited_by_count":178,"area":null},{"id":"2303.06296","title":"Stabilizing Transformer Training by Preventing Attention Entropy Collapse","created":"2023-03-11","cited_by_count":161,"area":null},{"id":"2206.03126","title":"Signal Propagation in Transformers: Theoretical Perspectives and the Role of Rank Collapse","created":"2022-06-07","cited_by_count":141,"area":null},{"id":"2210.06313","title":"The Lazy Neuron Phenomenon: On Emergence of Activation Sparsity in Transformers","created":"2022-10-12","cited_by_count":122,"area":null},{"id":"2211.08403","title":"REPAIR: REnormalizing Permuted Activations for Interpolation Repair","created":"2022-11-15","cited_by_count":120,"area":null},{"id":"2104.04975","title":"Scalable Marginal Likelihood Estimation for Model Selection in Deep Learning","created":"2021-04-11","cited_by_count":106,"area":null},{"id":"2104.07167","title":"Orthogonalizing Convolutional Layers with the Cayley Transform","created":"2021-04-14","cited_by_count":105,"area":null},{"id":"2110.13100","title":"Parameter Prediction for Unseen Deep Architectures","created":"2021-10-25","cited_by_count":89,"area":null},{"id":"2101.08692","title":"Characterizing signal propagation to close the performance gap in unnormalized ResNets","created":"2021-01-21","cited_by_count":88,"area":null},{"id":"2102.11972","title":"Do Transformer Modifications Transfer Across Implementations and Applications?","created":"2021-02-23","cited_by_count":87,"area":null},{"id":"2303.00980","title":"Learning to Grow Pretrained Models for Efficient Transformer Training","created":"2023-03-02","cited_by_count":83,"area":null},{"id":"2505.01618","title":"Don't be lazy: CompleteP enables compute-efficient deep transformers","created":"2025-05-02","cited_by_count":78,"area":null},{"id":"2304.13013","title":"Stable and low-precision training for large-scale vision-language models","created":"2023-04-25","cited_by_count":75,"area":"mllm"},{"id":"2503.10622","title":"Transformers without Normalization","created":"2025-03-13","cited_by_count":70,"area":null},{"id":"2309.16620","title":"Depthwise Hyperparameter Transfer in Residual Networks: Dynamics and Scaling Limit","created":"2023-09-28","cited_by_count":66,"area":null},{"id":"2211.06291","title":"Do Bayesian Neural Networks Need To Be Fully Stochastic?","created":"2022-11-11","cited_by_count":63,"area":null},{"id":"2206.08720","title":"Fast Finite Width Neural Tangent Kernel","created":"2022-06-17","cited_by_count":62,"area":null},{"id":"2202.11678","title":"Bayesian Model Selection, the Marginal Likelihood, and Generalization","created":"2022-02-23","cited_by_count":57,"area":null},{"id":"2111.05193","title":"A Survey on Green Deep Learning","created":"2021-11-08","cited_by_count":52,"area":null},{"id":"2312.16903","title":"Spike No More: Stabilizing the Pre-training of Large Language Models","created":"2023-12-28","cited_by_count":52,"area":null},{"id":"2106.13299","title":"Free-viewpoint Indoor Neural Relighting from Multi-view Stereo","created":"2021-06-24","cited_by_count":50,"area":null},{"id":"2311.00636","title":"Kronecker-Factored Approximate Curvature for Modern Neural Network Architectures","created":"2023-11-01","cited_by_count":49,"area":null},{"id":"2202.00089","title":"Understanding AdamW through Proximal Methods and Scale-Freeness","created":"2022-01-31","cited_by_count":46,"area":null},{"id":"2302.10322","title":"Deep Transformers without Shortcuts: Modifying Self-attention for Faithful Signal Propagation","created":"2023-02-20","cited_by_count":45,"area":null},{"id":"2102.08098","title":"GradInit: Learning to Initialize Neural Networks for Stable and Efficient Training","created":"2021-02-16","cited_by_count":44,"area":null},{"id":"2202.10638","title":"Invariance Learning in Deep Neural Networks with Differentiable Laplace Approximations","created":"2022-02-22","cited_by_count":42,"area":null},{"id":"2205.02973","title":"Large Scale Transfer Learning for Differentially Private Image Classification","created":"2022-05-06","cited_by_count":41,"area":null},{"id":"2301.13442","title":"Scaling laws for single-agent reinforcement learning","created":"2023-01-31","cited_by_count":41,"area":null},{"id":"2110.04369","title":"A Loss Curvature Perspective on Training Instability in Deep Learning","created":"2021-10-08","cited_by_count":40,"area":null},{"id":"2302.01107","title":"A Survey on Efficient Training of Transformers","created":"2023-02-02","cited_by_count":39,"area":null},{"id":"2102.06559","title":"Infinitely Deep Bayesian Neural Networks with Stochastic Differential Equations","created":"2021-02-12","cited_by_count":38,"area":null},{"id":"2311.01906","title":"Simplifying Transformer Blocks","created":"2023-11-03","cited_by_count":37,"area":null},{"id":"2604.07822","title":"Loop, Think, & Generalize: Implicit Reasoning in Recurrent-Depth Transformers","created":"2026-04-09","cited_by_count":37,"area":"lr"},{"id":"2111.07668","title":"Fast Axiomatic Attribution for Neural Networks","created":"2021-11-15","cited_by_count":35,"area":null},{"id":"2304.09856","title":"LipsFormer: Introducing Lipschitz Continuity to Vision Transformers","created":"2023-04-19","cited_by_count":34,"area":null},{"id":"2106.04013","title":"The Future is Log-Gaussian: ResNets and Their Infinite-Depth-and-Width Limit at Initialization","created":"2021-06-07","cited_by_count":33,"area":null},{"id":"2111.13293","title":"KNAS: Green Neural Architecture Search","created":"2021-11-26","cited_by_count":33,"area":null},{"id":"2107.12898","title":"StarEnhancer: Learning Real-Time and Style-Aware Image Enhancement","created":"2021-07-27","cited_by_count":32,"area":null},{"id":"2304.13207","title":"EverLight: Indoor-Outdoor Editable HDR Lighting Estimation","created":"2023-04-26","cited_by_count":32,"area":null},{"id":"2408.11804","title":"Approaching Deep Learning through the Spectral Dynamics of Weights","created":"2024-08-21","cited_by_count":31,"area":null},{"id":"2203.02645","title":"Acceleration of Federated Learning with Alleviated Forgetting in Local Training","created":"2022-03-05","cited_by_count":29,"area":null},{"id":"2206.08900","title":"Adapting the Linearised Laplace Model Evidence for Modern Deep Learning","created":"2022-06-17","cited_by_count":29,"area":null},{"id":"2303.04143","title":"Can We Scale Transformers to Predict Parameters of Diverse ImageNet Models?","created":"2023-03-07","cited_by_count":29,"area":null},{"id":"2104.11044","title":"Analyzing Monotonic Linear Interpolation in Neural Network Loss Landscapes","created":"2021-04-22","cited_by_count":28,"area":null},{"id":"2110.12661","title":"ZerO Initialization: Initializing Neural Networks with only Zeros and Ones","created":"2021-10-25","cited_by_count":28,"area":null},{"id":"2203.08120","title":"Deep Learning without Shortcuts: Shaping the Kernel with Tailored Rectifiers","created":"2022-03-15","cited_by_count":28,"area":null},{"id":"2501.06842","title":"SPAM: Spike-Aware Adam with Momentum Reset for Stable LLM Training","created":"2025-01-12","cited_by_count":28,"area":null},{"id":"2306.04226","title":"Normalization Layers Are All That Sharpness-Aware Minimization Needs","created":"2023-06-07","cited_by_count":27,"area":null},{"id":"2202.10583","title":"MineRL Diamond 2021 Competition: Overview, Results, and Lessons Learned","created":"2022-02-17","cited_by_count":26,"area":null},{"id":"2305.13291","title":"Materialistic: Selecting Similar Materials in Images","created":"2023-05-22","cited_by_count":26,"area":null},{"id":"2310.13545","title":"ScaleLong: Towards More Stable Training of Diffusion Model via Scaling Network Long Skip Connection","created":"2023-10-20","cited_by_count":25,"area":"diffusion"},{"id":"2110.06850","title":"Boosting the Certified Robustness of L-infinity Distance Nets","created":"2021-10-13","cited_by_count":24,"area":null},{"id":"2207.01156","title":"Removing Batch Normalization Boosts Adversarial Training","created":"2022-07-04","cited_by_count":24,"area":null},{"id":"2105.07205","title":"Rethinking Skip Connection with Layer Normalization in Transformers and ResNets","created":"2021-05-15","cited_by_count":22,"area":null},{"id":"2106.03839","title":"NTIRE 2021 Challenge on Burst Super-Resolution: Methods and Results","created":"2021-06-07","cited_by_count":22,"area":null},{"id":"2104.03093","title":"Spectral Analysis of the Neural Tangent Kernel for Deep Residual Networks","created":"2021-04-07","cited_by_count":21,"area":null},{"id":"2412.07224","title":"Parseval Regularization for Continual Reinforcement Learning","created":"2024-12-10","cited_by_count":21,"area":"deeprl"},{"id":"2307.06092","title":"Quantitative CLTs in Deep Neural Networks","created":"2023-07-12","cited_by_count":20,"area":null},{"id":"2309.01213","title":"Implicit regularization of deep residual networks towards neural ODEs","created":"2023-09-03","cited_by_count":20,"area":null},{"id":"2103.07013","title":"Large Batch Simulation for Deep Reinforcement Learning","created":"2021-03-12","cited_by_count":19,"area":null},{"id":"2210.06423","title":"Foundation Transformers","created":"2022-10-12","cited_by_count":19,"area":null},{"id":"2312.08399","title":"Principled Weight Initialization for Hypernetworks","created":"2023-12-13","cited_by_count":19,"area":null},{"id":"2206.06929","title":"Scaling ResNets in the Large-depth Regime","created":"2022-06-14","cited_by_count":18,"area":null},{"id":"2104.08215","title":"\"BNN - BN = ?\": Training Binary Neural Networks without Batch Normalization","created":"2021-04-16","cited_by_count":17,"area":null},{"id":"2206.11925","title":"Set Norm and Equivariant Skip Connections: Putting the Deep in Deep Sets","created":"2022-06-23","cited_by_count":17,"area":null},{"id":"2206.00330","title":"B2T Connection: Serving Stability and Performance in Deep Transformers","created":"2022-06-01","cited_by_count":16,"area":null},{"id":"2310.10699","title":"Reusing Pretrained Models by Multi-linear Operators for Efficient Training","created":"2023-10-16","cited_by_count":15,"area":null},{"id":"2304.12372","title":"Beyond the Pixel: a Photometrically Calibrated HDR Dataset for Luminance and Color Prediction","created":"2023-04-24","cited_by_count":14,"area":null},{"id":"2301.12549","title":"Unlocking Deterministic Robustness Certification on ImageNet","created":"2023-01-29","cited_by_count":13,"area":null},{"id":"2512.10938","title":"Stronger Normalization-Free Transformers","created":"2025-12-11","cited_by_count":13,"area":null},{"id":"2007.02701","title":"Scaling Imitation Learning in Minecraft","created":"2020-07-06","cited_by_count":12,"area":"offlinerl"},{"id":"2106.03743","title":"Proxy-Normalizing Activations to Match Batch Normalization while Removing Batch Dependence","created":"2021-06-07","cited_by_count":12,"area":null},{"id":"2202.06438","title":"Learning from Randomly Initialized Neural Network Features","created":"2022-02-13","cited_by_count":12,"area":null},{"id":"2506.10973","title":"Principled Approaches for Extending Neural Architectures to Function Spaces for Operator Learning","created":"2025-06-12","cited_by_count":12,"area":null},{"id":"2306.05879","title":"FedWon: Triumphing Multi-domain Federated Learning Without Normalization","created":"2023-06-09","cited_by_count":11,"area":null},{"id":"2410.04779","title":"Fast Training of Sinusoidal Neural Fields via Scaling Initialization","created":"2024-10-07","cited_by_count":11,"area":null},{"id":"2306.03968","title":"Stochastic Marginal Likelihood Gradients using Neural Tangent Kernels","created":"2023-06-06","cited_by_count":10,"area":null},{"id":"2402.05913","title":"Efficient Stagewise Pretraining via Progressive Subnetworks","created":"2024-02-08","cited_by_count":10,"area":null},{"id":"2405.19279","title":"Understanding and Minimising Outlier Features in Neural Network Training","created":"2024-05-29","cited_by_count":10,"area":null},{"id":"2606.18524","title":"On the Residual Scaling of Looped Transformers: Stability and Transferability","created":"2026-06-16","cited_by_count":10,"area":null},{"id":"2108.13320","title":"Neural HMMs are all you need (for high-quality attention-free TTS)","created":"2021-08-30","cited_by_count":8,"area":null},{"id":"2303.06530","title":"Making Batch Normalization Great in Federated Deep Learning","created":"2023-03-12","cited_by_count":8,"area":null},{"id":"2304.14766","title":"Hyperparameter Optimization through Neural Network Partitioning","created":"2023-04-28","cited_by_count":8,"area":null},{"id":"2403.12887","title":"Understanding the training of infinitely deep and wide ResNets with Conditional Optimal Transport","created":"2024-03-19","cited_by_count":8,"area":null},{"id":"2407.11910","title":"Benchmarking the Attribution Quality of Vision Models","created":"2024-07-16","cited_by_count":8,"area":null},{"id":"2106.13924","title":"Self-Attentive Ensemble Transformer: Representing Ensemble Interactions in Neural Networks for Earth System Models","created":"2021-06-21","cited_by_count":7,"area":null},{"id":"2206.01515","title":"Understanding Deep Learning via Decision Boundary","created":"2022-06-03","cited_by_count":7,"area":null},{"id":"2607.13491","title":"DeepLoop: Depth Scaling for Looped Transformers","created":"2026-07-15","cited_by_count":7,"area":null},{"id":"2112.02834","title":"A Generalized Zero-Shot Quantization of Deep Convolutional Neural Networks via Learned Weights Statistics","created":"2021-12-06","cited_by_count":6,"area":null},{"id":"2210.06143","title":"On the Importance of Gradient Norm in PAC-Bayesian Bounds","created":"2022-10-12","cited_by_count":6,"area":null},{"id":"2305.02790","title":"BranchNorm: Robustly Scaling Extremely Deep Transformers","created":"2023-05-04","cited_by_count":6,"area":null},{"id":"2410.18408","title":"Scale Propagation Network for Generalizable Depth Completion","created":"2024-10-24","cited_by_count":6,"area":null},{"id":"2109.08958","title":"AutoInit: Analytic Signal-Preserving Weight Initialization for Neural Networks","created":"2021-09-18","cited_by_count":5,"area":null}],"next_cursor":"100"}