{"total":140,"rows":[{"id":"2503.11651","title":"VGGT: Visual Geometry Grounded Transformer","created":"2025-03-14","cited_by_count":1539,"area":null},{"id":"2309.14322","title":"Small-scale proxies for large-scale Transformer training instabilities","created":"2023-09-25","cited_by_count":183,"area":null},{"id":"2312.06709","title":"AM-RADIO: Agglomerative Vision Foundation Model -- Reduce All Domains Into One","created":"2023-12-10","cited_by_count":169,"area":null},{"id":"2402.16788","title":"Why Transformers Need Adam: A Hessian Perspective","created":"2024-02-26","cited_by_count":124,"area":null},{"id":"2406.04267","title":"Transformers need glasses! Information over-squashing in language tasks","created":"2024-06-06","cited_by_count":91,"area":null},{"id":"2410.21676","title":"How Does Critical Batch Size Scale in Pre-training?","created":"2024-10-29","cited_by_count":73,"area":null},{"id":"2503.10622","title":"Transformers without Normalization","created":"2025-03-13","cited_by_count":70,"area":null},{"id":"2410.13835","title":"Active-Dormant Attention Heads: Mechanistically Demystifying Extreme-Token Phenomena in LLMs","created":"2024-10-17","cited_by_count":56,"area":null},{"id":"2312.16903","title":"Spike No More: Stabilizing the Pre-training of Large Language Models","created":"2023-12-28","cited_by_count":52,"area":null},{"id":"2306.17759","title":"The Shaped Transformer: Attention Models in the Infinite Depth-and-Width Limit","created":"2023-06-30","cited_by_count":47,"area":null},{"id":"2401.09865","title":"Improving fine-grained understanding in image-text pre-training","created":"2024-01-18","cited_by_count":44,"area":null},{"id":"2402.10198","title":"SAMformer: Unlocking the Potential of Transformers in Time Series Forecasting with Sharpness-Aware Minimization and Channel-Wise Attention","created":"2024-02-15","cited_by_count":44,"area":null},{"id":"2310.09971","title":"AMAGO: Scalable In-Context Reinforcement Learning for Adaptive Agents","created":"2023-10-15","cited_by_count":41,"area":null},{"id":"2503.09543","title":"PolyPythias: Stability and Outliers across Fifty Language Model Pre-Training Runs","created":"2025-03-12","cited_by_count":35,"area":null},{"id":"2406.06811","title":"Learning Continually by Spectral Regularization","created":"2024-06-10","cited_by_count":34,"area":null},{"id":"2502.02732","title":"Peri-LN: Revisiting Normalization Layer in the Transformer Architecture","created":"2025-02-04","cited_by_count":34,"area":null},{"id":"2501.06842","title":"SPAM: Spike-Aware Adam with Momentum Reset for Stable LLM Training","created":"2025-01-12","cited_by_count":28,"area":null},{"id":"2401.04301","title":"Setting the Record Straight on Transformer Oversmoothing","created":"2024-01-09","cited_by_count":21,"area":null},{"id":"2506.16640","title":"Long-Context Generalization with Sparse Attention","created":"2025-06-19","cited_by_count":21,"area":null},{"id":"2402.02098","title":"Self-attention Networks Localize When QK-eigenspectrum Concentrates","created":"2024-02-03","cited_by_count":19,"area":null},{"id":"2505.24333","title":"Two failure modes of deep transformers and how to avoid them: a unified theory of signal propagation at initialisation","created":"2025-05-30","cited_by_count":17,"area":null},{"id":"2310.12956","title":"Eureka-Moments in Transformers: Multi-Step Tasks Reveal Softmax Induced Optimization Problems","created":"2023-10-19","cited_by_count":14,"area":null},{"id":"2410.16682","title":"Methods of improving LLM training stability","created":"2024-10-22","cited_by_count":13,"area":null},{"id":"2506.04805","title":"Adaptive Preconditioners Trigger Loss Spikes in Adam","created":"2025-06-05","cited_by_count":13,"area":null},{"id":"2404.08634","title":"When Attention Collapses: How Degenerate Layers in LLMs Enable Smaller, Stronger Models","created":"2024-04-12","cited_by_count":12,"area":null},{"id":"2312.04234","title":"Graph Convolutions Enrich the Self-Attention in Transformers!","created":"2023-12-07","cited_by_count":11,"area":null},{"id":"2505.20524","title":"Towards Fully FP8 GEMM LLM Training at Scale","created":"2025-05-26","cited_by_count":11,"area":null},{"id":"2507.07814","title":"Pay Attention to Attention Distribution: A New Local Lipschitz Bound for Transformers","created":"2025-07-10","cited_by_count":11,"area":null},{"id":"2405.13985","title":"LookHere: Vision Transformers with Directed Attention Generalize and Extrapolate","created":"2024-05-22","cited_by_count":10,"area":null},{"id":"2405.19279","title":"Understanding and Minimising Outlier Features in Neural Network Training","created":"2024-05-29","cited_by_count":10,"area":null},{"id":"2406.15303","title":"AEM: Attention Entropy Maximization for Multiple Instance Learning based Whole Slide Image Classification","created":"2024-06-18","cited_by_count":10,"area":null},{"id":"2403.05490","title":"Poly-View Contrastive Learning","created":"2024-03-08","cited_by_count":9,"area":null},{"id":"2406.12719","title":"Exploring the Robustness of Language Models for Tabular Question Answering via Attention Analysis","created":"2024-06-18","cited_by_count":9,"area":null},{"id":"2501.14577","title":"ZETA: Leveraging Z-order Curves for Efficient Top-k Attention","created":"2025-01-24","cited_by_count":9,"area":null},{"id":"2505.21910","title":"Taming Transformer Without Using Learning Rate Warmup","created":"2025-05-28","cited_by_count":9,"area":null},{"id":"2510.09017","title":"Value-State Gated Attention for Mitigating Extreme-Token Phenomena in Transformers","created":"2025-10-10","cited_by_count":8,"area":null},{"id":"2511.11553","title":"Multistability of Self-Attention Dynamics in Transformers","created":"2025-11-14","cited_by_count":8,"area":null},{"id":"2402.15415","title":"Understanding Catastrophic Forgetting In LoRA via Mean-Field Attention Dynamics","created":"2024-02-23","cited_by_count":7,"area":null},{"id":"2411.00999","title":"Normalization Layer Per-Example Gradients are Sufficient to Predict Gradient Noise Scale in Transformers","created":"2024-11-01","cited_by_count":7,"area":null},{"id":"2508.00264","title":"Calibrated Language Models and How to Find Them with Label Smoothing","created":"2025-08-01","cited_by_count":7,"area":null},{"id":"2409.07787","title":"Stable Language Model Pre-training by Reducing Embedding Variability","created":"2024-09-12","cited_by_count":6,"area":null},{"id":"2502.16570","title":"Entropy-Lens: Uncovering Decision Strategies in LLMs","created":"2025-02-23","cited_by_count":6,"area":null},{"id":"2504.16275","title":"Quantum Doubly Stochastic Transformers","created":"2025-04-22","cited_by_count":6,"area":null},{"id":"2507.23632","title":"On the Expressiveness of Softmax Attention: A Recurrent Neural Network Perspective","created":"2025-07-31","cited_by_count":6,"area":null},{"id":"2512.21691","title":"Analyzing the Mechanism of Attention Collapse in VGGT from a Dynamics Perspective","created":"2025-12-25","cited_by_count":6,"area":null},{"id":"2402.10202","title":"Bridging Associative Memory and Probabilistic Modeling","created":"2024-02-15","cited_by_count":5,"area":null},{"id":"2410.05052","title":"Initialization of Large Language Models via Reparameterization to Mitigate Loss Spikes","created":"2024-10-07","cited_by_count":4,"area":null},{"id":"2412.21124","title":"Adaptive Batch Size Schedules for Distributed Training of Language Models with Data and Model Parallelism","created":"2024-12-30","cited_by_count":4,"area":null},{"id":"2502.14704","title":"Not All Data are Good Labels: On the Self-supervised Labeling for Time Series Forecasting","created":"2025-02-20","cited_by_count":4,"area":null},{"id":"2505.15325","title":"SoftHGNN: Soft Hypergraph Neural Networks for General Visual Recognition","created":"2025-05-21","cited_by_count":4,"area":null},{"id":"2602.05287","title":"Position: Universal Time Series Foundation Models Rest on a Category Error","created":"2026-02-05","cited_by_count":4,"area":null},{"id":"2602.10408","title":"Gated Normalization Removal and Scale Anchoring in Pre-Norm Transformers","created":"2026-02-11","cited_by_count":4,"area":null},{"id":"2402.02634","title":"Key-Graph Transformer for Image Restoration","created":"2024-02-04","cited_by_count":3,"area":null},{"id":"2408.13152","title":"Long-term Pre-training for Temporal Action Detection with Transformers","created":"2024-08-23","cited_by_count":3,"area":null},{"id":"2410.09637","title":"ReLU's Revival: On the Entropic Overload in Normalization-Free Large Language Models","created":"2024-10-12","cited_by_count":3,"area":null},{"id":"2410.13060","title":"AERO: Entropy-Guided Framework for Private LLM Inference","created":"2024-10-16","cited_by_count":3,"area":null},{"id":"2502.11066","title":"CARMA: Enhanced Compositionality in LLMs via Advanced Regularisation and Mutual Information Alignment","created":"2025-02-16","cited_by_count":3,"area":null},{"id":"2506.01562","title":"Unpacking Softmax: How Temperature Drives Representation Collapse, Compression, and Generalization","created":"2025-06-02","cited_by_count":3,"area":null},{"id":"2508.18387","title":"Integral Transformer: Denoising Attention, Not Too Much Not Too Little","created":"2025-08-25","cited_by_count":3,"area":null},{"id":"2509.07324","title":"Mitigating Attention Localization in Small Scale: Self-Attention Refinement via One-step Belief Propagation","created":"2025-09-09","cited_by_count":3,"area":null},{"id":"2511.21377","title":"Controlling changes to attention logits","created":"2025-11-26","cited_by_count":3,"area":null},{"id":"2602.18849","title":"Exact Attention Sensitivity and the Geometry of Transformer Stability","created":"2026-02-21","cited_by_count":3,"area":null},{"id":"2602.20433","title":"Disentangling Geometry, Performance, and Training in Language Models","created":"2026-02-24","cited_by_count":3,"area":null},{"id":"2603.07162","title":"Spectral Conditioning of Attention Improves Transformer Performance","created":"2026-03-07","cited_by_count":3,"area":null},{"id":"2603.14483","title":"Disentangling Dynamical Systems: Causal Representation Learning Meets Local Sparse Attention","created":"2026-03-15","cited_by_count":3,"area":null},{"id":"2603.23998","title":"Sparse Growing Transformer: Training-Time Sparse Depth Allocation via Progressive Attention Looping","created":"2026-03-25","cited_by_count":3,"area":null},{"id":"2604.00726","title":"Exploring Silent Data Corruption as a Reliability Challenge in LLM Training","created":"2026-04-01","cited_by_count":3,"area":null},{"id":"2604.15180","title":"AdaSplash-2: Faster Differentiable Sparse Attention","created":"2026-04-16","cited_by_count":3,"area":null},{"id":"2405.15903","title":"UnitNorm: Rethinking Normalization for Transformers in Time Series","created":"2024-05-24","cited_by_count":2,"area":null},{"id":"2410.04801","title":"Improving Image Clustering with Artifacts Attenuation via Inference-Time Attention Engineering","created":"2024-10-07","cited_by_count":2,"area":null},{"id":"2411.01443","title":"Activating Self-Attention for Multi-Scene Absolute Pose Regression","created":"2024-11-03","cited_by_count":2,"area":null},{"id":"2503.18337","title":"Coeff-Tuning: A Graph Filter Subspace View for Tuning Attention-Based Large Models","created":"2025-03-24","cited_by_count":2,"area":null},{"id":"2506.12903","title":"Variational Learning Finds Flatter Solutions at the Edge of Stability","created":"2025-06-15","cited_by_count":2,"area":null},{"id":"2508.00127","title":"Structured Transformations for Stable and Interpretable Neural Computation","created":"2025-07-31","cited_by_count":2,"area":null},{"id":"2510.14254","title":"Generalist vs Specialist Time Series Foundation Models: Investigating Potential Emergent Behaviors in Assessing Human Health Using PPG Signals","created":"2025-10-16","cited_by_count":2,"area":null},{"id":"2602.08216","title":"Entropy-Generated Attention Beyond Softmax and Entmax: Kaniadakis and Reciprocal-Symmetric Abe Operators","created":"2026-02-09","cited_by_count":2,"area":null},{"id":"2602.18851","title":"Rank-Aware Spectral Bounds on Attention Logits for Stable Low-Precision Training","created":"2026-02-21","cited_by_count":2,"area":null},{"id":"2603.06248","title":"Gradient Flow Polarizes Softmax Outputs towards Low-Entropy Solutions","created":"2026-03-06","cited_by_count":2,"area":null},{"id":"2604.07925","title":"Sinkhorn doubly stochastic attention rank decay analysis","created":"2026-04-09","cited_by_count":2,"area":null},{"id":"2604.13123","title":"Spectral Entropy Collapse as a Phase Transition in Delayed Generalisation: An Interventional and Predictive Framework for Grokkin","created":"2026-04-13","cited_by_count":2,"area":null},{"id":"2604.17324","title":"Capacity-Controlled Global Attention for Graph Transformers","created":"2026-04-19","cited_by_count":2,"area":null},{"id":"2605.04279","title":"Gradient Flow Structure and Quantitative Dynamics of Multi-Head Self-Attention","created":"2026-05-05","cited_by_count":2,"area":null},{"id":"2605.10504","title":"Learning Less Is More: Premature Upper-Layer Attention Specialization Hurts Language Model Pretraining","created":"2026-05-11","cited_by_count":2,"area":null},{"id":"2606.28116","title":"Mechanism-Driven Monitors for Preemptive Detection of LLM Training Instability","created":"2026-06-26","cited_by_count":2,"area":null},{"id":"2312.06182","title":"Why \"classic\" Transformers are shallow and how to make them go deep","created":"2023-12-11","cited_by_count":1,"area":null},{"id":"2312.17372","title":"Beyond PID Controllers: PPO with Neuralized PID Policy for Proton Beam Intensity Control in Mu2e","created":"2023-12-28","cited_by_count":1,"area":null},{"id":"2505.04738","title":"SetONet: A Set-Based Operator Network for Solving PDEs with Variable-Input Sampling","created":"2025-05-07","cited_by_count":1,"area":null},{"id":"2506.11305","title":"Don't Pay Attention","created":"2025-06-12","cited_by_count":1,"area":null},{"id":"2507.17501","title":"DNT: a Deeply Normalized Transformer that can be trained by Momentum SGD","created":"2025-07-23","cited_by_count":1,"area":null},{"id":"2508.04925","title":"Why Attention Fails: A Taxonomy of Faults in Attention-Based Neural Networks","created":"2025-08-06","cited_by_count":1,"area":null},{"id":"2510.13832","title":"Entropy Meets Importance: A Unified Head Importance-Entropy Score for Stable and Efficient Transformer Pruning","created":"2025-10-10","cited_by_count":1,"area":null},{"id":"2511.20698","title":"On the Role of Hidden States of Modern Hopfield Network in Transformer","created":"2025-11-24","cited_by_count":1,"area":null},{"id":"2602.10410","title":"LUCID: Attention with Preconditioned Representations","created":"2026-02-11","cited_by_count":1,"area":null},{"id":"2604.00199","title":"QUEST: A robust attention formulation using query-modulated spherical attention","created":"2026-03-31","cited_by_count":1,"area":null},{"id":"2604.01683","title":"Invertible Query-Key Coupling Composes with Attention Mechanisms","created":"2026-04-02","cited_by_count":1,"area":null},{"id":"2604.11890","title":"Subcritical Signal Propagation at Initialization in Normalization-Free Transformers","created":"2026-04-13","cited_by_count":1,"area":null},{"id":"2605.09271","title":"Shaping Schema via Language Representation as the Next Frontier for LLM Intelligence Expanding","created":"2026-05-10","cited_by_count":1,"area":null},{"id":"2605.21292","title":"Unbiased Gradients, Moving Stability Boundaries: Exact Mini-Batch Geometry in Linear Self-Attention","created":"2026-05-20","cited_by_count":1,"area":null},{"id":"2606.24752","title":"Can Scale Save Us From Plasticity Loss in Large Language Models?","created":"2026-06-23","cited_by_count":1,"area":null},{"id":"2608.08922","title":"Clustered Attractor Manifolds and Dynamical Condensation in Self-Attention","created":"2026-08-09","cited_by_count":1,"area":null}],"next_cursor":"100"}