{"total":231,"rows":[{"id":"2103.17239","title":"Going deeper with Image Transformers","created":"2021-03-31","cited_by_count":764,"area":null},{"id":"2106.04554","title":"A Survey of Transformers","created":"2021-06-08","cited_by_count":401,"area":null},{"id":"2102.06171","title":"High-Performance Large-Scale Image Recognition Without Normalization","created":"2021-02-11","cited_by_count":360,"area":null},{"id":"2203.03466","title":"Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer","created":"2022-03-07","cited_by_count":288,"area":null},{"id":"2103.03841","title":"Generating Images with Sparse Representations","created":"2021-03-05","cited_by_count":266,"area":null},{"id":"2312.03863","title":"Efficient Large Language Models: A Survey","created":"2023-12-06","cited_by_count":186,"area":null},{"id":"2203.00555","title":"DeepNet: Scaling Transformers to 1,000 Layers","created":"2022-03-01","cited_by_count":178,"area":null},{"id":"2303.06296","title":"Stabilizing Transformer Training by Preventing Attention Entropy Collapse","created":"2023-03-11","cited_by_count":161,"area":null},{"id":"2206.03126","title":"Signal Propagation in Transformers: Theoretical Perspectives and the Role of Rank Collapse","created":"2022-06-07","cited_by_count":141,"area":null},{"id":"2103.11511","title":"MoViNets: Mobile Video Networks for Efficient Video Recognition","created":"2021-03-21","cited_by_count":135,"area":null},{"id":"2402.16788","title":"Why Transformers Need Adam: A Hessian Perspective","created":"2024-02-26","cited_by_count":124,"area":null},{"id":"2210.06313","title":"The Lazy Neuron Phenomenon: On Emergence of Activation Sparsity in Transformers","created":"2022-10-12","cited_by_count":122,"area":null},{"id":"2304.03208","title":"Cerebras-GPT: Open Compute-Optimal Language Models Trained on the Cerebras Wafer-Scale Cluster","created":"2023-04-06","cited_by_count":109,"area":null},{"id":"2101.08692","title":"Characterizing signal propagation to close the performance gap in unnormalized ResNets","created":"2021-01-21","cited_by_count":88,"area":null},{"id":"2102.11972","title":"Do Transformer Modifications Transfer Across Implementations and Applications?","created":"2021-02-23","cited_by_count":87,"area":null},{"id":"2110.09456","title":"NormFormer: Improved Transformer Pretraining with Extra Normalization","created":"2021-10-18","cited_by_count":83,"area":null},{"id":"2203.00867","title":"Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encoding","created":"2022-03-02","cited_by_count":83,"area":null},{"id":"2301.03313","title":"BQ-NCO: Bisimulation Quotienting for Efficient Neural Combinatorial Optimization","created":"2023-01-09","cited_by_count":82,"area":null},{"id":"2306.13089","title":"GIMLET: A Unified Graph-Text Model for Instruction-Based Molecule Zero-Shot Learning","created":"2023-05-28","cited_by_count":70,"area":null},{"id":"2410.01131","title":"nGPT: Normalized Transformer with Representation Learning on the Hypersphere","created":"2024-10-01","cited_by_count":70,"area":null},{"id":"2503.10622","title":"Transformers without Normalization","created":"2025-03-13","cited_by_count":70,"area":null},{"id":"2305.19587","title":"Towards Omni-generalizable Neural Methods for Vehicle Routing Problems","created":"2023-05-31","cited_by_count":69,"area":null},{"id":"2309.16620","title":"Depthwise Hyperparameter Transfer in Residual Networks: Dynamics and Scaling Limit","created":"2023-09-28","cited_by_count":66,"area":null},{"id":"2603.15031","title":"Attention Residuals","created":"2026-03-16","cited_by_count":65,"area":null},{"id":"2105.12787","title":"Self-Supervised Bug Detection and Repair","created":"2021-05-26","cited_by_count":59,"area":null},{"id":"2110.01765","title":"Rapid training of deep neural networks without skip connections or normalization layers using Deep Kernel Shaping","created":"2021-10-05","cited_by_count":52,"area":null},{"id":"2309.16137","title":"Context-I2W: Mapping Images to Context-dependent Words for Accurate Zero-Shot Composed Image Retrieval","created":"2023-09-28","cited_by_count":52,"area":null},{"id":"2401.11673","title":"MVSFormer++: Revealing the Devil in Transformer's Details for Multi-View Stereo","created":"2024-01-22","cited_by_count":50,"area":null},{"id":"2205.01138","title":"Transformers in Time-series Analysis: A Tutorial","created":"2022-04-28","cited_by_count":46,"area":null},{"id":"2302.10322","title":"Deep Transformers without Shortcuts: Modifying Self-attention for Faithful Signal Propagation","created":"2023-02-20","cited_by_count":45,"area":null},{"id":"2102.08098","title":"GradInit: Learning to Initialize Neural Networks for Stable and Efficient Training","created":"2021-02-16","cited_by_count":44,"area":null},{"id":"2203.02557","title":"UVCGAN: UNet Vision Transformer cycle-consistent GAN for unpaired image-to-image translation","created":"2022-03-04","cited_by_count":42,"area":null},{"id":"2406.15079","title":"GOAL: A Generalist Combinatorial Optimization Agent Learner","created":"2024-06-21","cited_by_count":42,"area":null},{"id":"2307.06440","title":"No Train No Gain: Revisiting Efficient Training Algorithms For Transformer-based Language Models","created":"2023-07-12","cited_by_count":41,"area":null},{"id":"2302.01107","title":"A Survey on Efficient Training of Transformers","created":"2023-02-02","cited_by_count":39,"area":null},{"id":"2203.14509","title":"Automated Progressive Learning for Efficient Training of Vision Transformers","created":"2022-03-28","cited_by_count":37,"area":null},{"id":"2311.01906","title":"Simplifying Transformer Blocks","created":"2023-11-03","cited_by_count":37,"area":null},{"id":"2104.08400","title":"Structure-Aware Abstractive Conversation Summarization via Discourse and Action Graphs","created":"2021-04-16","cited_by_count":36,"area":null},{"id":"2410.01405","title":"On Expressive Power of Looped Transformers: Theoretical Analysis and Enhancement via Timestep Encoding","created":"2024-10-02","cited_by_count":36,"area":null},{"id":"2209.05310","title":"On the Factory Floor: ML Engineering for Industrial-Scale Ads Recommendation Models","created":"2022-09-12","cited_by_count":34,"area":null},{"id":"2301.02819","title":"ExcelFormer: A neural network surpassing GBDTs on tabular data","created":"2023-01-07","cited_by_count":34,"area":null},{"id":"2304.09856","title":"LipsFormer: Introducing Lipschitz Continuity to Vision Transformers","created":"2023-04-19","cited_by_count":34,"area":null},{"id":"2304.14802","title":"ResiDual: Transformer with Dual Residual Connections","created":"2023-04-28","cited_by_count":31,"area":null},{"id":"2104.00430","title":"Latent Space Data Assimilation by using Deep Learning","created":"2021-04-01","cited_by_count":30,"area":null},{"id":"2110.12661","title":"ZerO Initialization: Initializing Neural Networks with only Zeros and Ones","created":"2021-10-25","cited_by_count":28,"area":null},{"id":"2203.08120","title":"Deep Learning without Shortcuts: Shaping the Kernel with Tailored Rectifiers","created":"2022-03-15","cited_by_count":28,"area":null},{"id":"2306.03009","title":"Using Sequences of Life-events to Predict Human Lives","created":"2023-06-05","cited_by_count":27,"area":null},{"id":"2403.01493","title":"ConvTimeNet: A Deep Hierarchical Fully Convolutional Model for Multivariate Time Series Analysis","created":"2024-03-03","cited_by_count":27,"area":null},{"id":"2103.06498","title":"3D Human Pose, Shape and Texture from Low-Resolution Images and Videos","created":"2021-03-11","cited_by_count":26,"area":null},{"id":"2107.03134","title":"MedGPT: Medical Concept Prediction from Clinical Narratives","created":"2021-07-07","cited_by_count":24,"area":null},{"id":"2207.01156","title":"Removing Batch Normalization Boosts Adversarial Training","created":"2022-07-04","cited_by_count":24,"area":null},{"id":"2105.07205","title":"Rethinking Skip Connection with Layer Normalization in Transformers and ResNets","created":"2021-05-15","cited_by_count":22,"area":null},{"id":"2205.13349","title":"Learning What and Where: Disentangling Location and Identity Tracking Without Supervision","created":"2022-05-26","cited_by_count":21,"area":null},{"id":"2210.05950","title":"ZITS++: Image Inpainting by Improving the Incremental Transformer on Structural Priors","created":"2022-10-12","cited_by_count":21,"area":null},{"id":"2302.09818","title":"FormerTime: Hierarchical Multi-Scale Representations for Multivariate Time Series Classification","created":"2023-02-20","cited_by_count":21,"area":null},{"id":"2311.02265","title":"Not all layers are equally as important: Every Layer Counts BERT","created":"2023-11-03","cited_by_count":21,"area":null},{"id":"2105.06987","title":"Scaling Ensemble Distribution Distillation to Many Classes with Proxy Targets","created":"2021-05-14","cited_by_count":19,"area":null},{"id":"2110.06081","title":"On Expressivity and Trainability of Quadratic Networks","created":"2021-10-12","cited_by_count":19,"area":null},{"id":"2402.02098","title":"Self-attention Networks Localize When QK-eigenspectrum Concentrates","created":"2024-02-03","cited_by_count":19,"area":null},{"id":"2402.10093","title":"MIM-Refiner: A Contrastive Learning Boost from Intermediate Pre-Trained Representations","created":"2024-02-15","cited_by_count":19,"area":null},{"id":"2107.00961","title":"ResIST: Layer-Wise Decomposition of ResNets for Distributed Training","created":"2021-07-02","cited_by_count":18,"area":null},{"id":"2206.06929","title":"Scaling ResNets in the Large-depth Regime","created":"2022-06-14","cited_by_count":18,"area":null},{"id":"2405.17272","title":"DPN: Decoupling Partition and Navigation for Neural Solvers of Min-max Vehicle Routing Problems","created":"2024-05-27","cited_by_count":18,"area":null},{"id":"2104.08215","title":"\"BNN - BN = ?\": Training Binary Neural Networks without Batch Normalization","created":"2021-04-16","cited_by_count":17,"area":null},{"id":"2105.12245","title":"Scaling Properties of Deep Residual Networks","created":"2021-05-25","cited_by_count":16,"area":null},{"id":"2111.05948","title":"Scaling ASR Improves Zero and Few Shot Learning","created":"2021-11-10","cited_by_count":16,"area":null},{"id":"2208.01837","title":"Learning Prior Feature and Attention Enhanced Image Inpainting","created":"2022-08-03","cited_by_count":16,"area":null},{"id":"2209.03177","title":"Morphology-preserving Autoregressive 3D Generative Modelling of the Brain","created":"2022-09-07","cited_by_count":15,"area":null},{"id":"2303.00579","title":"Are More Layers Beneficial to Graph Transformers?","created":"2023-03-01","cited_by_count":15,"area":null},{"id":"2310.10699","title":"Reusing Pretrained Models by Multi-linear Operators for Efficient Training","created":"2023-10-16","cited_by_count":15,"area":null},{"id":"2404.09010","title":"MMA-DFER: MultiModal Adaptation of unimodal models for Dynamic Facial Expression Recognition in-the-wild","created":"2024-04-13","cited_by_count":14,"area":null},{"id":"2404.09466","title":"Scoring Time Intervals using Non-Hierarchical Transformer For Automatic Piano Transcription","created":"2024-04-15","cited_by_count":14,"area":null},{"id":"2107.11114","title":"A comparison of combined data assimilation and machine learning methods for offline and online model error correction","created":"2021-07-23","cited_by_count":13,"area":null},{"id":"2203.03691","title":"HyperMixer: An MLP-based Low Cost Alternative to Transformers","created":"2022-03-07","cited_by_count":13,"area":null},{"id":"2204.08479","title":"Inductive Biases for Object-Centric Representations in the Presence of Complex Textures","created":"2022-04-18","cited_by_count":13,"area":null},{"id":"2301.13310","title":"Alternating Updates for Efficient Transformers","created":"2023-01-30","cited_by_count":13,"area":null},{"id":"2310.15422","title":"G2-MonoDepth: A General Framework of Generalized Depth Inference from Monocular RGB+X Data","created":"2023-10-24","cited_by_count":13,"area":null},{"id":"2512.10938","title":"Stronger Normalization-Free Transformers","created":"2025-12-11","cited_by_count":13,"area":null},{"id":"2202.06438","title":"Learning from Randomly Initialized Neural Network Features","created":"2022-02-13","cited_by_count":12,"area":null},{"id":"2506.22696","title":"Residual Matrix Transformers: Scaling the Size of the Residual Stream","created":"2025-06-28","cited_by_count":12,"area":null},{"id":"2203.09445","title":"Image Super-Resolution With Deep Variational Autoencoders","created":"2022-03-17","cited_by_count":11,"area":null},{"id":"2411.19305","title":"LD-EnSF: Synergizing Latent Dynamics with Ensemble Score Filters for Fast Data Assimilation with Sparse Observations","created":"2024-11-28","cited_by_count":11,"area":null},{"id":"2207.14467","title":"GTrans: Grouping and Fusing Transformer Layers for Neural Machine Translation","created":"2022-07-29","cited_by_count":10,"area":null},{"id":"2402.05913","title":"Efficient Stagewise Pretraining via Progressive Subnetworks","created":"2024-02-08","cited_by_count":10,"area":null},{"id":"2405.19279","title":"Understanding and Minimising Outlier Features in Neural Network Training","created":"2024-05-29","cited_by_count":10,"area":null},{"id":"2510.06954","title":"From Condensation to Rank Collapse: A Two-Stage Analysis of Transformer Training Dynamics","created":"2025-10-08","cited_by_count":10,"area":null},{"id":"2602.08064","title":"SiameseNorm: Breaking the Barrier to Reconciling Pre/Post-Norm","created":"2026-02-08","cited_by_count":10,"area":null},{"id":"2606.18524","title":"On the Residual Scaling of Looped Transformers: Stability and Transferability","created":"2026-06-16","cited_by_count":10,"area":null},{"id":"2309.09574","title":"Latent assimilation with implicit neural representations for unknown dynamics","created":"2023-09-18","cited_by_count":9,"area":null},{"id":"2412.02890","title":"EvRT-DETR: Latent Space Adaptation of Image Detectors for Event-based Vision","created":"2024-12-03","cited_by_count":9,"area":null},{"id":"2505.21910","title":"Taming Transformer Without Using Learning Rate Warmup","created":"2025-05-28","cited_by_count":9,"area":null},{"id":"2111.12143","title":"Critical Initialization of Wide and Deep Neural Networks through Partial Jacobians: General Theory and Applications","created":"2021-11-23","cited_by_count":8,"area":null},{"id":"2601.21942","title":"Clustering in Deep Stochastic Transformers","created":"2026-01-29","cited_by_count":8,"area":null},{"id":"2206.05893","title":"Deploying Convolutional Networks on Untrusted Platforms Using 2D Holographic Reduced Representations","created":"2022-06-13","cited_by_count":7,"area":null},{"id":"2208.10771","title":"Learning an Efficient Multimodal Depth Completion Model","created":"2022-08-23","cited_by_count":7,"area":null},{"id":"2306.11023","title":"PINQI: An End-to-End Physics-Informed Approach to Learned Quantitative MRI Reconstruction","created":"2023-06-19","cited_by_count":7,"area":null},{"id":"2308.07761","title":"NeFL: Nested Model Scaling for Federated Learning with System Heterogeneous Clients","created":"2023-08-15","cited_by_count":7,"area":null},{"id":"2406.09160","title":"Beyond the Frontier: Predicting Unseen Walls from Occupancy Grids by Learning from Floor Plans","created":"2024-06-13","cited_by_count":7,"area":null},{"id":"2605.18855","title":"Delta Attention Residuals","created":"2026-05-13","cited_by_count":7,"area":null},{"id":"2607.13491","title":"DeepLoop: Depth Scaling for Looped Transformers","created":"2026-07-15","cited_by_count":7,"area":null}],"next_cursor":"100"}