{"total":53,"rows":[{"id":"2410.01104","title":"Softmax is not Enough (for Sharp Size Generalisation)","created":"2024-10-01","cited_by_count":46,"area":null},{"id":"2502.06415","title":"Systematic Outliers in Large Language Models","created":"2025-02-10","cited_by_count":33,"area":null},{"id":"2603.05498","title":"The Spike, the Sparse and the Sink: Anatomy of Massive Activations and Attention Sinks","created":"2026-03-05","cited_by_count":27,"area":null},{"id":"2502.18845","title":"Sliding Window Attention Training for Efficient Large Language Models","created":"2025-02-26","cited_by_count":24,"area":null},{"id":"2601.22966","title":"A Unified View of Attention and Residual Sinks: Outlier-Driven Rescaling is Essential for Transformer Training","created":"2026-01-30","cited_by_count":23,"area":null},{"id":"2410.01537","title":"Attention layers provably solve single-location regression","created":"2024-10-02","cited_by_count":17,"area":null},{"id":"2505.01996","title":"Always Skip Attention","created":"2025-05-04","cited_by_count":13,"area":null},{"id":"2409.19315","title":"Analog In-Memory Computing Attention Mechanism for Fast and Energy-Efficient Large Language Models","created":"2024-09-28","cited_by_count":11,"area":null},{"id":"2510.09017","title":"Value-State Gated Attention for Mitigating Extreme-Token Phenomena in Transformers","created":"2025-10-10","cited_by_count":8,"area":null},{"id":"2510.04008","title":"RACE Attention: A Strictly Linear-Time Attention Layer for Training on Outrageously Large Contexts","created":"2025-10-05","cited_by_count":5,"area":null},{"id":"2602.05230","title":"ZeroS: Zero-Sum Linear Attention for Efficient Transformers","created":"2026-02-05","cited_by_count":5,"area":null},{"id":"2605.25704","title":"PowLU: An Activation Function for Stable Pre-Training of LLMs","created":"2026-05-25","cited_by_count":4,"area":null},{"id":"2502.18277","title":"Self-Adjust Softmax","created":"2025-02-25","cited_by_count":3,"area":null},{"id":"2506.01562","title":"Unpacking Softmax: How Temperature Drives Representation Collapse, Compression, and Generalization","created":"2025-06-02","cited_by_count":3,"area":null},{"id":"2507.04239","title":"Scaling Context Requires Rethinking Attention","created":"2025-07-06","cited_by_count":3,"area":null},{"id":"2507.20453","title":"Your Attention Matters: to Improve Model Robustness to Noise and Spurious Correlations","created":"2025-07-28","cited_by_count":3,"area":null},{"id":"2512.01370","title":"PRISMA: Improving the Accuracy-Latency Frontier of Diffusion-based PDE Solvers Using Physics-Informed Spectral Attention","created":"2025-12-01","cited_by_count":3,"area":null},{"id":"2601.00919","title":"Attention Needs to Focus: A Unified Perspective on Attention Allocation","created":"2026-01-01","cited_by_count":3,"area":null},{"id":"2607.04319","title":"Legible-by-Construction: Attention and End-to-End Transformers","created":"2026-07-05","cited_by_count":3,"area":null},{"id":"2509.09424","title":"ENSI: Efficient Non-Interactive Secure Inference for Large Language Models","created":"2025-09-11","cited_by_count":2,"area":null},{"id":"2512.11411","title":"Sliced ReLU attention: Quasi-linear contextual expressivity via sorting","created":"2025-12-12","cited_by_count":2,"area":null},{"id":"2603.06248","title":"Gradient Flow Polarizes Softmax Outputs towards Low-Entropy Solutions","created":"2026-03-06","cited_by_count":2,"area":null},{"id":"2605.08504","title":"A Single Layer to Explain Them All:Understanding Massive Activations in Large Language Models","created":"2026-05-08","cited_by_count":2,"area":null},{"id":"2501.13428","title":"Softplus Attention with Re-weighting Boosts Length Extrapolation in Large Language Models","created":"2025-01-23","cited_by_count":1,"area":null},{"id":"2501.16215","title":"Smarter Together: Combining Large Language Models and Small Models for Physiological Signals Visual Inspection","created":"2025-01-27","cited_by_count":1,"area":null},{"id":"2507.05829","title":"Intra-DP: A High Performance Collaborative Inference System for Mobile Edge Computing","created":"2025-07-08","cited_by_count":1,"area":null},{"id":"2511.08888","title":"Weaver: Kronecker Product Approximations of Spatiotemporal Attention for Traffic Network Forecasting","created":"2025-11-12","cited_by_count":1,"area":null},{"id":"2602.02707","title":"Every Bit Counts: A Theoretical Study of Precision-Expressivity Tradeoffs in Quantized Transformers","created":"2026-02-02","cited_by_count":1,"area":null},{"id":"2604.01178","title":"Screening Is Enough","created":"2026-04-01","cited_by_count":1,"area":null},{"id":"2605.06611","title":"The Structural Origin of Attention Sink: Variance Discrepancy, Super Neurons, and Dimension Disparity","created":"2026-05-07","cited_by_count":1,"area":null},{"id":"2605.18870","title":"Multi-Headed Transformer Architectures as Time-dependent Wasserstein Gradient Flows","created":"2026-05-15","cited_by_count":1,"area":null},{"id":"2605.20798","title":"Most Transformer Modifications Still Do Not Transfer at 1-3B: A 2020-2026 Update to Narang et al. (2021) with Downstream Evaluation and a Noise Floor","created":"2026-05-20","cited_by_count":1,"area":null},{"id":"2610.00564","title":"Attention Kernels for Learning Maps Between Heavy-Tailed Measures","created":"2026-09-30","cited_by_count":1,"area":null},{"id":"2507.03922","title":"Dynamic Injection of Entity Knowledge into Dense Retrievers","created":"2025-07-05","cited_by_count":0,"area":null},{"id":"2510.11789","title":"Minimax Rates for Learning Pairwise Interactions in Attention-Style Models","created":"2025-10-13","cited_by_count":0,"area":null},{"id":"2603.00021","title":"From Global to Local: Learning Context-Aware Graph Representations for Document Classification and Summarization","created":"2026-02-03","cited_by_count":0,"area":null},{"id":"2604.09512","title":"Integrated electro-optic attention nonlinearities for transformers","created":"2026-04-10","cited_by_count":0,"area":null},{"id":"2604.27124","title":"Better Models, Faster Training: Sigmoid Attention for single-cell Foundation Models","created":"2026-04-29","cited_by_count":0,"area":null},{"id":"2605.06501","title":"Cubit: Token Mixer with Kernel Ridge Regression","created":"2026-05-07","cited_by_count":0,"area":null},{"id":"2606.02332","title":"Forget Attention: Importance-Aware Attention Is All You Need","created":"2026-06-01","cited_by_count":0,"area":null},{"id":"2606.08105","title":"A Unifying View of Attention Sinks: From Mechanisms to Architectural Interventions","created":"2026-06-06","cited_by_count":0,"area":null},{"id":"2606.12478","title":"Boltzmann Attention: Learnable Ising Couplings for Cooperative Attention","created":"2026-06-10","cited_by_count":0,"area":null},{"id":"2606.30139","title":"Warranted Attention: Learning What to Pass from Attention to Prediction","created":"2026-06-29","cited_by_count":0,"area":null},{"id":"2607.17342","title":"STAR: Skeletal Token Alignment and Rearrangement for Interaction Recognition","created":"2026-07-19","cited_by_count":0,"area":null},{"id":"2607.20484","title":"The Devil is in the Spectrum: Mitigating Representation Collapse in LLMs via Topologically Regularized Side-Path","created":"2026-06-02","cited_by_count":0,"area":null},{"id":"2607.23634","title":"Variational-Ising-Attention:Tailored Attention Matters for Science","created":"2026-07-26","cited_by_count":0,"area":null},{"id":"2608.20172","title":"Ask Self, Ask Others: Relation Is All You Need","created":"2026-08-20","cited_by_count":0,"area":null},{"id":"2608.21174","title":"From Attention Masks to Inert Zero-Vector Tokens: OAttention and O-Closure for Token Dynamics","created":"2026-08-21","cited_by_count":0,"area":null},{"id":"2609.33591","title":"Pretraining Transformers with Quantized Softmax in Attention","created":"2026-09-27","cited_by_count":0,"area":null},{"id":"2609.39188","title":"Attention Function as an Intrinsic Inductive Bias: How Models' Behavior Diverges in Novel Contexts","created":"2026-09-30","cited_by_count":0,"area":null},{"id":"2610.00049","title":"Fast Polynomial Transcendentals for LLMs","created":"2026-09-03","cited_by_count":0,"area":null},{"id":"2610.03873","title":"Streaming Multi-Track Timeline Control for 3D Human Motion Generation","created":"2026-10-02","cited_by_count":0,"area":null},{"id":"2610.05488","title":"Underscoring the Problem: Why Softpick Fails at Initialization","created":"2026-10-04","cited_by_count":0,"area":null}],"next_cursor":null}