{"total":164,"rows":[{"id":"2504.02732","title":"Why do LLMs attend to the first token?","created":"2025-04-03","cited_by_count":127,"area":null},{"id":"2407.11046","title":"A Survey on LoRA of Large Language Models","created":"2024-07-08","cited_by_count":67,"area":null},{"id":"2510.06477","title":"Attention Sinks and Compression Valleys in LLMs are Two Sides of the Same Coin","created":"2025-10-07","cited_by_count":52,"area":null},{"id":"2405.18781","title":"On the Role of Attention Masks and LayerNorm in Transformers","created":"2024-05-29","cited_by_count":48,"area":null},{"id":"2312.14820","title":"How Smooth Is Attention?","created":"2023-12-22","cited_by_count":44,"area":null},{"id":"2408.01367","title":"Transformers are Universal In-context Learners","created":"2024-08-02","cited_by_count":43,"area":null},{"id":"2410.23228","title":"Emergence of meta-stable clustering in mean-field transformer models","created":"2024-10-30","cited_by_count":39,"area":null},{"id":"2411.04990","title":"Clustering in Causal Attention Masking","created":"2024-11-07","cited_by_count":32,"area":null},{"id":"2402.03902","title":"A phase transition between positional and semantic learning in a solvable model of dot-product attention","created":"2024-02-06","cited_by_count":30,"area":null},{"id":"2512.01868","title":"The Mean-Field Dynamics of Transformers","created":"2025-12-01","cited_by_count":30,"area":null},{"id":"2411.04551","title":"Measure-to-measure interpolation using Transformers","created":"2024-11-07","cited_by_count":29,"area":null},{"id":"2504.14697","title":"Quantitative Clustering in Mean-Field Transformer Models","created":"2025-04-20","cited_by_count":26,"area":null},{"id":"2509.25040","title":"A multiscale analysis of mean-field transformers in the moderate interaction regime","created":"2025-09-29","cited_by_count":25,"area":null},{"id":"2403.02579","title":"Geometric Dynamics of Signal Propagation Predict Trainability of Transformers","created":"2024-03-05","cited_by_count":24,"area":null},{"id":"2406.12616","title":"Learning diffusion at lightspeed","created":"2024-06-18","cited_by_count":22,"area":null},{"id":"2402.04084","title":"Provably learning a multi-head attention layer","created":"2024-02-06","cited_by_count":21,"area":null},{"id":"2510.05554","title":"Critical attention scaling in long-context transformers","created":"2025-10-07","cited_by_count":20,"area":null},{"id":"2407.16826","title":"SINDER: Repairing the Singular Defects of DINOv2","created":"2024-07-23","cited_by_count":18,"area":null},{"id":"2501.00658","title":"Understanding and Mitigating Bottlenecks of State Space Models through the Lens of Recency and Over-smoothing","created":"2024-12-31","cited_by_count":18,"area":"ssm"},{"id":"2405.18273","title":"Synchronization on circles and spheres with nonlinear interactions","created":"2024-05-28","cited_by_count":17,"area":null},{"id":"2501.10573","title":"The Intrinsic Dimension of Prompts in Internal Representations of Large Language Models","created":"2025-01-17","cited_by_count":17,"area":"mechinterp"},{"id":"2505.24333","title":"Two failure modes of deep transformers and how to avoid them: a unified theory of signal propagation at initialisation","created":"2025-05-30","cited_by_count":17,"area":null},{"id":"2412.02682","title":"The Asymptotic Behavior of Attention in Transformers","created":"2024-12-03","cited_by_count":16,"area":null},{"id":"2502.00919","title":"Attention Sinks: A 'Catch, Tag, Release' Mechanism for Embeddings","created":"2025-02-02","cited_by_count":16,"area":null},{"id":"2504.01002","title":"Token embeddings violate the manifold hypothesis","created":"2025-04-01","cited_by_count":16,"area":null},{"id":"2407.01602","title":"Clustering in pure-attention hardmax transformers and its role in sentiment analysis","created":"2024-06-26","cited_by_count":15,"area":null},{"id":"2506.10959","title":"Understanding In-Context Learning on Structured Manifolds: Bridging Attention to Kernel Methods","created":"2025-06-12","cited_by_count":15,"area":null},{"id":"2402.18551","title":"Implicit Optimization Bias of Next-Token Prediction in Linear Models","created":"2024-02-28","cited_by_count":14,"area":null},{"id":"2507.22857","title":"Synchronization of mean-field models on the circle","created":"2025-07-30","cited_by_count":12,"area":null},{"id":"2510.20094","title":"On the Structure of Stationary Solutions to McKean-Vlasov Equations with Applications to Noisy Transformers","created":"2025-10-23","cited_by_count":12,"area":null},{"id":"2504.17836","title":"Learning Enhanced Ensemble Filters","created":"2025-04-24","cited_by_count":11,"area":null},{"id":"2405.15926","title":"Dissecting the Interplay of Attention Paths in a Statistical Mechanics Theory of Transformers","created":"2024-05-24","cited_by_count":9,"area":null},{"id":"2503.03025","title":"Hierarchical Refinement: Optimal Transport to Infinity and Beyond","created":"2025-03-04","cited_by_count":9,"area":null},{"id":"2510.22026","title":"Normalization in Attention Dynamics","created":"2025-10-24","cited_by_count":9,"area":null},{"id":"2505.13499","title":"Optimal Control for Transformer Architectures: Enhancing Generalization, Robustness and Efficiency","created":"2025-05-16","cited_by_count":8,"area":null},{"id":"2511.11553","title":"Multistability of Self-Attention Dynamics in Transformers","created":"2025-11-14","cited_by_count":8,"area":null},{"id":"2512.14366","title":"Optimizing Rank for High-Fidelity Implicit Neural Representations","created":"2025-12-16","cited_by_count":8,"area":null},{"id":"2512.16762","title":"NRGPT: An Energy-based Alternative for GPT","created":"2025-12-18","cited_by_count":8,"area":null},{"id":"2601.21366","title":"Perceptrons and localization of attention's mean-field landscape","created":"2026-01-29","cited_by_count":8,"area":null},{"id":"2601.21942","title":"Clustering in Deep Stochastic Transformers","created":"2026-01-29","cited_by_count":8,"area":null},{"id":"2402.15415","title":"Understanding Catastrophic Forgetting In LoRA via Mean-Field Attention Dynamics","created":"2024-02-23","cited_by_count":7,"area":null},{"id":"2406.13762","title":"Unveiling the Hidden Structure of Self-Attention via Kernel Principal Component Analysis","created":"2024-06-19","cited_by_count":7,"area":null},{"id":"2410.16295","title":"Universal Approximation of Mean-Field Models via Transformers","created":"2024-10-06","cited_by_count":7,"area":null},{"id":"2411.10555","title":"Low-Rank Optimal Transport through Factor Relaxation with Latent Coupling","created":"2024-11-15","cited_by_count":7,"area":null},{"id":"2501.18793","title":"OT-Transformer: A Continuous-time Transformer Architecture with Optimal Transport Regularization","created":"2025-01-30","cited_by_count":7,"area":null},{"id":"2509.25611","title":"Transformers through the lens of support-preserving maps between measures","created":"2025-09-30","cited_by_count":6,"area":null},{"id":"2512.06797","title":"Optimal and Diffusion Transports in Machine Learning","created":"2025-12-07","cited_by_count":6,"area":null},{"id":"2505.19458","title":"Recurrent Self-Attention Dynamics: An Energy-Agnostic Perspective from Jacobians","created":"2025-05-26","cited_by_count":5,"area":null},{"id":"2506.03790","title":"Attention-Only Transformers via Unrolled Subspace Denoising","created":"2025-06-04","cited_by_count":5,"area":null},{"id":"2507.02748","title":"Linear Attention with Global Context: A Multipole Attention Mechanism for Vision and Physics","created":"2025-07-03","cited_by_count":5,"area":null},{"id":"2603.22301","title":"Latent Semantic Manifolds in Large Language Models","created":"2026-03-17","cited_by_count":5,"area":null},{"id":"2605.10931","title":"Quantifying Concentration Phenomena of Mean-Field Transformers in the Low-Temperature Regime","created":"2026-05-11","cited_by_count":5,"area":null},{"id":"2605.14258","title":"Dynamics of the Transformer Residual Stream: Coupling Spectral Geometry to Network Topology","created":"2026-05-14","cited_by_count":5,"area":null},{"id":"2412.09080","title":"On the number of modes of Gaussian kernel density estimators","created":"2024-12-12","cited_by_count":4,"area":null},{"id":"2506.15881","title":"T-SHRED: Symbolic Regression for Regularization and Model Discovery with Transformer Shallow Recurrent Decoders","created":"2025-06-18","cited_by_count":4,"area":null},{"id":"2603.06187","title":"Random Quadratic Form on a Sphere: Synchronization by Common Noise","created":"2026-03-06","cited_by_count":4,"area":null},{"id":"2605.06873","title":"One Operator for Many Densities: Amortized Approximation of Conditioning by Neural Operators","created":"2026-05-07","cited_by_count":4,"area":null},{"id":"2607.01520","title":"The risk of KV cache compression","created":"2026-07-01","cited_by_count":4,"area":"inference"},{"id":"2408.03816","title":"Early Prediction of Causes (not Effects) in Healthcare by Long-Term Clinical Time Series Forecasting","created":"2024-08-07","cited_by_count":3,"area":null},{"id":"2410.18858","title":"Bilinear Sequence Regression: A Model for Learning from Long Sequences of High-dimensional Tokens","created":"2024-10-24","cited_by_count":3,"area":null},{"id":"2412.18288","title":"Towards understanding how attention mechanism works in deep learning","created":"2024-12-24","cited_by_count":3,"area":null},{"id":"2502.07553","title":"Transformers Provably Learn Sparse XOR with Polylogarithmic Parameters","created":"2025-02-11","cited_by_count":3,"area":null},{"id":"2503.15421","title":"Probing the topology of the space of tokens with structured prompts","created":"2025-03-19","cited_by_count":3,"area":null},{"id":"2507.13540","title":"Provable Low-Frequency Bias of In-Context Learning of Representations","created":"2025-07-17","cited_by_count":3,"area":null},{"id":"2511.01202","title":"Forget BIT, It is All about TOKEN: Towards Semantic Information Theory for LLMs","created":"2025-11-03","cited_by_count":3,"area":null},{"id":"2512.10656","title":"Token Sample Complexity of Attention","created":"2025-12-11","cited_by_count":3,"area":null},{"id":"2512.11784","title":"Softmax as Linear Attention in the Large-Prompt Regime: a Measure-based Perspective","created":"2025-12-12","cited_by_count":3,"area":null},{"id":"2603.09571","title":"An Optimal Control Approach To Transformer Training","created":"2026-03-10","cited_by_count":3,"area":null},{"id":"2605.08505","title":"Scaling Limits of Long-Context Transformers","created":"2026-05-08","cited_by_count":3,"area":null},{"id":"2605.16747","title":"Propagation of Chaos in Contextual Flow Maps","created":"2026-05-16","cited_by_count":3,"area":null},{"id":"2605.22010","title":"Uniform-in-Time Weak Propagation-of-Chaos in Shallow Neural Networks","created":"2026-05-21","cited_by_count":3,"area":null},{"id":"2607.22367","title":"Interior interpretability with attention rollout: contraction and propagation profiles in Transformers","created":"2026-07-24","cited_by_count":3,"area":null},{"id":"2410.03292","title":"Demystifying the Token Dynamics of Deep Selective State Space Models","created":"2024-10-04","cited_by_count":2,"area":"ssm"},{"id":"2502.02270","title":"Exact Sequence Interpolation with Transformers","created":"2025-02-04","cited_by_count":2,"area":null},{"id":"2506.23551","title":"A unified framework for establishing the universal approximation of transformer-type architectures","created":"2025-06-30","cited_by_count":2,"area":null},{"id":"2509.15958","title":"Localmax dynamics for attention in transformers and its asymptotic behavior","created":"2025-09-19","cited_by_count":2,"area":null},{"id":"2510.03989","title":"A Mathematical Explanation of Transformers","created":"2025-10-05","cited_by_count":2,"area":null},{"id":"2510.14027","title":"Context-Selective State Space Models: Feedback is All You Need","created":"2025-10-15","cited_by_count":2,"area":"ssm"},{"id":"2510.23912","title":"Key and Value Weights Are Probably All You Need: On the Necessity of the Query, Key, Value weight Triplet in Self-Attention Transformers","created":"2025-10-27","cited_by_count":2,"area":null},{"id":"2512.03058","title":"Dynamical Properties of Tokens in Self-Attention and Effects of Positional Encoding","created":"2025-11-25","cited_by_count":2,"area":null},{"id":"2601.23236","title":"YuriiFormer: A Suite of Nesterov-Accelerated Transformers","created":"2026-01-30","cited_by_count":2,"area":null},{"id":"2602.00217","title":"Dispersion Loss Counteracts Embedding Condensation and Improves Generalization in Small Language Models","created":"2026-01-30","cited_by_count":2,"area":null},{"id":"2603.04430","title":"Flowers: A Warp Drive for Neural PDE Solvers","created":"2026-02-17","cited_by_count":2,"area":null},{"id":"2604.01577","title":"Exploration of Fast-Slow Latent Recurrence for Train-Short, Test-Long Generalization","created":"2026-04-02","cited_by_count":2,"area":null},{"id":"2604.16288","title":"Phase transitions in Doi-Onsager, Noisy Transformer, and other multimodal models","created":"2026-04-17","cited_by_count":2,"area":null},{"id":"2605.04279","title":"Gradient Flow Structure and Quantitative Dynamics of Multi-Head Self-Attention","created":"2026-05-05","cited_by_count":2,"area":null},{"id":"2605.06723","title":"When Does a Language Model Commit? A Finite-Answer Theory of Pre-Verbalization Commitment","created":"2026-05-07","cited_by_count":2,"area":"lr"},{"id":"2605.17968","title":"Function graph transformers universally approximate operators between function spaces","created":"2026-05-18","cited_by_count":2,"area":null},{"id":"2605.28075","title":"Measure-to-measure Regression with Transformers","created":"2026-05-27","cited_by_count":2,"area":null},{"id":"2403.08739","title":"The Garden of Forking Paths: Observing Dynamic Parameters Distribution in Large Language Models","created":"2024-03-13","cited_by_count":1,"area":null},{"id":"2408.15946","title":"Sigma Flows for Image and Data Labeling and Learning Structured Prediction","created":"2024-08-28","cited_by_count":1,"area":null},{"id":"2505.17282","title":"Attention with Trained Embeddings Provably Selects Important Tokens","created":"2025-05-22","cited_by_count":1,"area":null},{"id":"2506.11465","title":"RollingQ: Reviving the Cooperation Dynamics in Multimodal Transformer","created":"2025-06-13","cited_by_count":1,"area":null},{"id":"2508.12787","title":"Wavy Transformer","created":"2025-08-18","cited_by_count":1,"area":null},{"id":"2509.01685","title":"Preconditioned Regularized Wasserstein Proximal Sampling","created":"2025-09-01","cited_by_count":1,"area":null},{"id":"2510.16356","title":"Sparse Transformer Architectures via Regularized Wasserstein Proximal Operator with $L_1$ Prior","created":"2025-10-18","cited_by_count":1,"area":null},{"id":"2602.20624","title":"Physics-based phenomenological characterization of cross-modal bias in multimodal models","created":"2026-02-24","cited_by_count":1,"area":null},{"id":"2603.16535","title":"SympFormer: Accelerated attention blocks via Inertial Dynamics on Density Manifolds","created":"2026-03-17","cited_by_count":1,"area":null},{"id":"2604.01683","title":"Invertible Query-Key Coupling Composes with Attention Mechanisms","created":"2026-04-02","cited_by_count":1,"area":null},{"id":"2604.15632","title":"Algebraic Invariants of Lightning Self-Attention","created":"2026-04-17","cited_by_count":1,"area":null}],"next_cursor":"100"}