{"total":36,"rows":[{"id":"2504.02732","title":"Why do LLMs attend to the first token?","created":"2025-04-03","cited_by_count":127,"area":null},{"id":"2502.01951","title":"On the Emergence of Position Bias in Transformers","created":"2025-02-04","cited_by_count":59,"area":null},{"id":"2510.06477","title":"Attention Sinks and Compression Valleys in LLMs are Two Sides of the Same Coin","created":"2025-10-07","cited_by_count":52,"area":null},{"id":"2502.01563","title":"Massive Values in Self-Attention Modules are the Key to Contextual Knowledge Understanding","created":"2025-02-03","cited_by_count":47,"area":null},{"id":"2603.05498","title":"The Spike, the Sparse and the Sink: Anatomy of Massive Activations and Attention Sinks","created":"2026-03-05","cited_by_count":27,"area":null},{"id":"2504.20966","title":"Softpick: No Attention Sink, No Massive Activations with Rectified Softmax","created":"2025-04-29","cited_by_count":22,"area":null},{"id":"2502.00919","title":"Attention Sinks: A 'Catch, Tag, Release' Mechanism for Embeddings","created":"2025-02-02","cited_by_count":16,"area":null},{"id":"2411.13476","title":"When Precision Meets Position: BFloat16 Breaks Down RoPE in Long-Context Training","created":"2024-11-20","cited_by_count":15,"area":null},{"id":"2504.01928","title":"Is the Reversal Curse a Binding Problem? Uncovering Limitations of Transformers from a Basic Generalization Failure","created":"2025-04-02","cited_by_count":13,"area":null},{"id":"2506.19697","title":"Outlier-Safe Pre-Training for Robust 4-Bit Quantization of Large Language Models","created":"2025-06-24","cited_by_count":13,"area":null},{"id":"2505.11739","title":"ZeroTuning: Unlocking the Initial Token's Power to Enhance Large Language Models Without Training","created":"2025-05-16","cited_by_count":12,"area":null},{"id":"2603.11487","title":"Attention Sinks Are Provably Necessary in Softmax Transformers: Evidence from Trigger-Conditional Tasks","created":"2026-03-12","cited_by_count":11,"area":null},{"id":"2604.10098","title":"Attention Sink in Transformers: A Survey on Utilization, Interpretation, and Mitigation","created":"2026-04-11","cited_by_count":11,"area":null},{"id":"2510.10276","title":"Lost in the Middle: An Emergent Property from Information Retrieval Demands in LLMs","created":"2025-10-11","cited_by_count":9,"area":null},{"id":"2504.03889","title":"Identifying and Evaluating Inactive Heads in Pretrained LLMs","created":"2025-04-04","cited_by_count":8,"area":null},{"id":"2510.09017","title":"Value-State Gated Attention for Mitigating Extreme-Token Phenomena in Transformers","created":"2025-10-10","cited_by_count":8,"area":null},{"id":"2602.16837","title":"A Structural Theory of Position Bias in Transformers","created":"2026-02-18","cited_by_count":5,"area":null},{"id":"2603.17771","title":"Attention Sinks Induce Gradient Sinks: Massive Activations as Gradient Regulators in Transformers","created":"2026-03-18","cited_by_count":3,"area":null},{"id":"2509.16875","title":"Towards Interpretable and Efficient Attention: Compressing All by Contracting a Few","created":"2025-09-21","cited_by_count":2,"area":null},{"id":"2603.06248","title":"Gradient Flow Polarizes Softmax Outputs towards Low-Entropy Solutions","created":"2026-03-06","cited_by_count":2,"area":null},{"id":"2606.02288","title":"Massive Spikes in LLMs are Bias Vectors: Mechanistic Uncovering and Spike-Free Quantization","created":"2026-06-01","cited_by_count":2,"area":null},{"id":"2602.01893","title":"Geometric Analysis of Token Selection in Multi-Head Attention","created":"2026-02-02","cited_by_count":1,"area":null},{"id":"2605.08453","title":"Sink vs. diagonal patterns as mechanisms for attention switch and oversmoothing prevention","created":"2026-05-08","cited_by_count":1,"area":null},{"id":"2510.11789","title":"Minimax Rates for Learning Pairwise Interactions in Attention-Style Models","created":"2025-10-13","cited_by_count":0,"area":null},{"id":"2601.03424","title":"Spectral Archaeology: The Causal Topology of Model Evolution","created":"2026-01-06","cited_by_count":0,"area":null},{"id":"2604.11912","title":"How Transformers Learn to Plan via Multi-Token Prediction","created":"2026-04-13","cited_by_count":0,"area":null},{"id":"2604.14722","title":"A Mechanistic Account of Attention Sinks in GPT-2: One Circuit, Broader Implications for Mitigation","created":"2026-04-16","cited_by_count":0,"area":null},{"id":"2605.16468","title":"Mechanistically Interpretable Neural Encoding Reveals Fine-Grained Functional Selectivity in Human Visual Cortex","created":"2026-05-15","cited_by_count":0,"area":null},{"id":"2606.07604","title":"Contribution Weights: A Geometrical Analysis of Self-Attention Transformers","created":"2026-05-29","cited_by_count":0,"area":null},{"id":"2606.08105","title":"A Unifying View of Attention Sinks: From Mechanisms to Architectural Interventions","created":"2026-06-06","cited_by_count":0,"area":null},{"id":"2609.22005","title":"Abstention and Noise Filtering: Two Missing Primitives of Softmax Attention","created":"2026-09-18","cited_by_count":0,"area":null},{"id":"2609.23585","title":"Global Ranks Survive, Selected Heads Shift: BOS-Sink Topology under 4-bit Weight-Only Quantization","created":"2026-09-20","cited_by_count":0,"area":null},{"id":"2609.32704","title":"CoWindow Attention: Full Causal Coverage Is a Collective Property","created":"2026-09-26","cited_by_count":0,"area":null},{"id":"2609.37879","title":"Retrieval Capacity of Self-Attention Under Competition","created":"2026-09-29","cited_by_count":0,"area":null},{"id":"2609.38832","title":"Scaling Parameter and Context in Attention: Native Sparse Attention from Mixture-of-Head","created":"2026-09-30","cited_by_count":0,"area":null},{"id":"2610.16745","title":"Before Attention Sinks Form: Hidden-State Geometry Reveals and Modulates Their Formation","created":"2026-10-12","cited_by_count":0,"area":null}],"next_cursor":null}