{"total":111,"rows":[{"id":"2502.02013","title":"Layer by Layer: Uncovering Hidden Representations in Language Models","created":"2025-02-04","cited_by_count":311,"area":null},{"id":"2504.02732","title":"Why do LLMs attend to the first token?","created":"2025-04-03","cited_by_count":127,"area":null},{"id":"2502.01951","title":"On the Emergence of Position Bias in Transformers","created":"2025-02-04","cited_by_count":59,"area":null},{"id":"2410.13835","title":"Active-Dormant Attention Heads: Mechanistically Demystifying Extreme-Token Phenomena in LLMs","created":"2024-10-17","cited_by_count":56,"area":null},{"id":"2510.06477","title":"Attention Sinks and Compression Valleys in LLMs are Two Sides of the Same Coin","created":"2025-10-07","cited_by_count":52,"area":null},{"id":"2506.08010","title":"Vision Transformers Don't Need Trained Registers","created":"2025-06-09","cited_by_count":41,"area":null},{"id":"2508.02546","title":"What are you sinking? A geometric approach on attention sink","created":"2025-08-04","cited_by_count":28,"area":null},{"id":"2603.05498","title":"The Spike, the Sparse and the Sink: Anatomy of Massive Activations and Attention Sinks","created":"2026-03-05","cited_by_count":27,"area":null},{"id":"2412.16545","title":"Attention Entropy is a Key Factor: An Analysis of Parallel Context Encoding with Full-attention-based Pre-trained Language Models","created":"2024-12-21","cited_by_count":26,"area":null},{"id":"2502.18845","title":"Sliding Window Attention Training for Efficient Large Language Models","created":"2025-02-26","cited_by_count":24,"area":null},{"id":"2601.22966","title":"A Unified View of Attention and Residual Sinks: Outlier-Driven Rescaling is Essential for Transformer Training","created":"2026-01-30","cited_by_count":23,"area":null},{"id":"2504.20966","title":"Softpick: No Attention Sink, No Massive Activations with Rectified Softmax","created":"2025-04-29","cited_by_count":22,"area":null},{"id":"2505.18132","title":"BiggerGait: Unlocking Gait Recognition with Layer-wise Representations from Large Vision Models","created":"2025-05-23","cited_by_count":20,"area":null},{"id":"2502.00919","title":"Attention Sinks: A 'Catch, Tag, Release' Mechanism for Embeddings","created":"2025-02-02","cited_by_count":16,"area":null},{"id":"2503.08908","title":"Interpreting the Repeated Token Phenomenon in Large Language Models","created":"2025-03-11","cited_by_count":16,"area":null},{"id":"2411.13476","title":"When Precision Meets Position: BFloat16 Breaks Down RoPE in Long-Context Training","created":"2024-11-20","cited_by_count":15,"area":null},{"id":"2505.11739","title":"ZeroTuning: Unlocking the Initial Token's Power to Enhance Large Language Models Without Training","created":"2025-05-16","cited_by_count":12,"area":null},{"id":"2507.07814","title":"Pay Attention to Attention Distribution: A New Local Lipschitz Bound for Transformers","created":"2025-07-10","cited_by_count":11,"area":null},{"id":"2510.22603","title":"Mitigating Attention Sinks and Massive Activations in Audio-Visual Speech Recognition with LLMs","created":"2025-10-26","cited_by_count":11,"area":null},{"id":"2603.11487","title":"Attention Sinks Are Provably Necessary in Softmax Transformers: Evidence from Trigger-Conditional Tasks","created":"2026-03-12","cited_by_count":11,"area":null},{"id":"2507.16018","title":"Artifacts and Attention Sinks: Structured Approximations for Efficient Vision Transformers","created":"2025-07-21","cited_by_count":10,"area":null},{"id":"2510.10276","title":"Lost in the Middle: An Emergent Property from Information Retrieval Demands in LLMs","created":"2025-10-11","cited_by_count":9,"area":null},{"id":"2605.23889","title":"HorizonStream: Long-Horizon Attention for Streaming 3D Reconstruction","created":"2026-05-22","cited_by_count":9,"area":null},{"id":"2504.03889","title":"Identifying and Evaluating Inactive Heads in Pretrained LLMs","created":"2025-04-04","cited_by_count":8,"area":null},{"id":"2510.04236","title":"Scaling Sequence-to-Sequence Generative Neural Rendering","created":"2025-10-05","cited_by_count":8,"area":null},{"id":"2510.09017","title":"Value-State Gated Attention for Mitigating Extreme-Token Phenomena in Transformers","created":"2025-10-10","cited_by_count":8,"area":null},{"id":"2601.19895","title":"Post-LayerNorm Is Back: Stable, ExpressivE, and Deep","created":"2026-01-27","cited_by_count":7,"area":null},{"id":"2503.15450","title":"SkyLadder: Better and Faster Pretraining via Context Window Scheduling","created":"2025-03-19","cited_by_count":6,"area":null},{"id":"2502.09647","title":"Unveiling Simplicities of Attention: Adaptive Long-Context Head Identification","created":"2025-02-11","cited_by_count":5,"area":null},{"id":"2505.17083","title":"Scale-invariant Attention","created":"2025-05-20","cited_by_count":5,"area":null},{"id":"2508.14067","title":"Punctuation and Predicates in Language Models","created":"2025-08-11","cited_by_count":5,"area":null},{"id":"2511.09146","title":"DoPE: Denoising Rotary Position Embedding","created":"2025-11-12","cited_by_count":5,"area":null},{"id":"2508.03668","title":"CTR-Sink: Attention Sink for Language Models in Click-Through Rate Prediction","created":"2025-08-05","cited_by_count":4,"area":null},{"id":"2512.22213","title":"On the Existence and Behavior of Secondary Attention Sinks","created":"2025-12-22","cited_by_count":4,"area":null},{"id":"2410.13060","title":"AERO: Entropy-Guided Framework for Private LLM Inference","created":"2024-10-16","cited_by_count":3,"area":null},{"id":"2504.17562","title":"When Does Metadata Conditioning (NOT) Work for Language Model Pre-Training? A Study with Context-Free Grammars","created":"2025-04-24","cited_by_count":3,"area":null},{"id":"2505.24179","title":"SALE : Low-bit Estimation for Efficient Sparse Attention in Long-context LLM Prefilling","created":"2025-05-30","cited_by_count":3,"area":null},{"id":"2507.04239","title":"Scaling Context Requires Rethinking Attention","created":"2025-07-06","cited_by_count":3,"area":null},{"id":"2507.04404","title":"LayerCake: Token-Aware Contrastive Decoding within Large Language Model Layers","created":"2025-07-06","cited_by_count":3,"area":null},{"id":"2601.00919","title":"Attention Needs to Focus: A Unified Perspective on Attention Allocation","created":"2026-01-01","cited_by_count":3,"area":null},{"id":"2602.23057","title":"Affine-Scaled Attention: Towards Flexible and Stable Transformer Attention","created":"2026-02-26","cited_by_count":3,"area":null},{"id":"2603.06591","title":"What Makes Position Zero Special? A Mechanistic Study of Position Zero Attention Sinks in LLMs","created":"2026-02-04","cited_by_count":3,"area":null},{"id":"2603.17771","title":"Attention Sinks Induce Gradient Sinks: Massive Activations as Gradient Regulators in Transformers","created":"2026-03-18","cited_by_count":3,"area":null},{"id":"2605.12922","title":"When Attention Closes: How LLMs Lose the Thread in Multi-Turn Interaction","created":"2026-05-13","cited_by_count":3,"area":null},{"id":"2507.09875","title":"Function Induction and Task Generalization: An Interpretability Study with Off-by-One Addition","created":"2025-07-14","cited_by_count":2,"area":null},{"id":"2509.23595","title":"Timber: Training-free Instruct Model Refining with Base via Effective Rank","created":"2025-09-28","cited_by_count":2,"area":null},{"id":"2602.04163","title":"BPDQ: Bit-Plane Decomposition Quantization on a Variable Grid for Large Language Models","created":"2026-02-04","cited_by_count":2,"area":null},{"id":"2602.18145","title":"Detecting Contextual Hallucinations in LLMs with Frequency-Aware Attention","created":"2026-02-20","cited_by_count":2,"area":null},{"id":"2603.06248","title":"Gradient Flow Polarizes Softmax Outputs towards Low-Entropy Solutions","created":"2026-03-06","cited_by_count":2,"area":null},{"id":"2604.05688","title":"Attention Editing: A Versatile Framework for Cross-Architecture Attention Conversion","created":"2026-04-07","cited_by_count":2,"area":null},{"id":"2604.10027","title":"SinkTrack: Attention Sink based Context Anchoring for Large Language Models","created":"2026-04-11","cited_by_count":2,"area":null},{"id":"2605.08504","title":"A Single Layer to Explain Them All:Understanding Massive Activations in Large Language Models","created":"2026-05-08","cited_by_count":2,"area":null},{"id":"2605.17887","title":"Attention Sinks and Outliers in Attention Residuals","created":"2026-05-18","cited_by_count":2,"area":null},{"id":"2606.02288","title":"Massive Spikes in LLMs are Bias Vectors: Mechanistic Uncovering and Spike-Free Quantization","created":"2026-06-01","cited_by_count":2,"area":null},{"id":"2607.07678","title":"How Data Shapes RoPE Frequency Usage: From Positional Scale Matching to Length Generalization","created":"2026-07-08","cited_by_count":2,"area":null},{"id":"2503.01832","title":"Rotary Offset Features in Large Language Models","created":"2025-03-03","cited_by_count":1,"area":null},{"id":"2505.08739","title":"Probability Consistency in Large Language Models: Theoretical Foundations Meet Empirical Discrepancies","created":"2025-05-13","cited_by_count":1,"area":null},{"id":"2510.26771","title":"STaMP: Sequence Transformation and Mixed Precision for Low-Precision Activation Quantization","created":"2025-10-30","cited_by_count":1,"area":null},{"id":"2512.09182","title":"Understanding the Failure Modes of Transformers through the Lens of Graph Neural Networks","created":"2025-12-09","cited_by_count":1,"area":null},{"id":"2602.01893","title":"Geometric Analysis of Token Selection in Multi-Head Attention","created":"2026-02-02","cited_by_count":1,"area":null},{"id":"2603.09616","title":"Surgical Repair of Collapsed Attention Heads in ALiBi Transformers","created":"2026-03-10","cited_by_count":1,"area":null},{"id":"2604.01025","title":"Fast and Accurate Probing of In-Training LLMs' Downstream Performances","created":"2026-04-01","cited_by_count":1,"area":null},{"id":"2605.04421","title":"FLUID: Continuous-Time Hyperconnected Sparse Transformer for Sink-Free Learning","created":"2026-05-06","cited_by_count":1,"area":null},{"id":"2605.06611","title":"The Structural Origin of Attention Sink: Variance Discrepancy, Super Neurons, and Dimension Disparity","created":"2026-05-07","cited_by_count":1,"area":null},{"id":"2605.08453","title":"Sink vs. diagonal patterns as mechanisms for attention switch and oversmoothing prevention","created":"2026-05-08","cited_by_count":1,"area":null},{"id":"2605.17279","title":"Rover: Context-aware Conflict Resolution with LLM","created":"2026-05-17","cited_by_count":1,"area":null},{"id":"2606.20743","title":"Massive Activations Are Architecturally Robust: A Controlled Scratch/Commitment Residual Stream Test","created":"2026-06-17","cited_by_count":1,"area":null},{"id":"2606.24752","title":"Can Scale Save Us From Plasticity Loss in Large Language Models?","created":"2026-06-23","cited_by_count":1,"area":null},{"id":"2608.11805","title":"Hybrid Gated Attention","created":"2026-08-12","cited_by_count":1,"area":null},{"id":"2608.14712","title":"Which Question Is Your Attention Metric Answering? Attention Rows as Compositional Data","created":"2026-08-11","cited_by_count":1,"area":null},{"id":"2608.21315","title":"Prompt-Model Interaction Reaches the Fixed Points: A deterministic, task-free structural readout -- and the factorizations of it that failed","created":"2026-08-21","cited_by_count":1,"area":null},{"id":"2504.04238","title":"Sensitivity Meets Sparsity: The Impact of Extremely Sparse Parameter Patterns on Theory-of-Mind of Large Language Models","created":"2025-04-05","cited_by_count":0,"area":null},{"id":"2507.08214","title":"Depth-Sequence Transformer (DST) for Segment-Specific ICA Calcification Mapping on Non-Contrast CT","created":"2025-07-10","cited_by_count":0,"area":null},{"id":"2510.22752","title":"Beyond Semantics: How Temporal Biases Shape Retrieval in Transformer and State-Space Models","created":"2025-10-26","cited_by_count":0,"area":null},{"id":"2511.20279","title":"SelfMOTR: Revisiting MOTR with Self-Generating Detection Priors","created":"2025-11-25","cited_by_count":0,"area":null},{"id":"2511.21613","title":"Beyond URLs: Metadata Diversity and Position for Efficient LLM Pretraining","created":"2025-11-26","cited_by_count":0,"area":null},{"id":"2512.20661","title":"Calibrating Transformer Attention via Task-Space Sensitivity Feedback","created":"2025-12-19","cited_by_count":0,"area":null},{"id":"2601.14522","title":"On the Runway Cascade of Transformers for Language Modeling","created":"2026-01-20","cited_by_count":0,"area":null},{"id":"2603.28773","title":"UltRAG: a Universal Simple Scalable Recipe for Knowledge Graph RAG","created":"2026-01-28","cited_by_count":0,"area":null},{"id":"2604.02973","title":"Exploring Motion-Language Alignment for Text-driven Motion Generation","created":"2026-04-03","cited_by_count":0,"area":null},{"id":"2604.10697","title":"Attention Sinks as Internal Signals for Hallucination Detection in Large Language Models","created":"2026-04-12","cited_by_count":0,"area":null},{"id":"2605.09213","title":"Kinetic theory for Transformers and the lost-in-the-middle phenomenon","created":"2026-05-09","cited_by_count":0,"area":null},{"id":"2605.10503","title":"SLASH the Sink: Sharpening Structural Attention Inside LLMs","created":"2026-05-11","cited_by_count":0,"area":null},{"id":"2605.11447","title":"Conditional Memory Enhanced Item Representation for Generative Recommendation","created":"2026-05-12","cited_by_count":0,"area":null},{"id":"2605.22372","title":"ASAP: Attention Sink Anchored Pruning","created":"2026-05-21","cited_by_count":0,"area":null},{"id":"2606.03712","title":"When Graph Tokens Sink: A Mechanistic Analysis of Graph Language Models","created":"2026-06-02","cited_by_count":0,"area":null},{"id":"2606.06160","title":"Where does Absolute Position come from in decoder-only Transformers?","created":"2026-06-04","cited_by_count":0,"area":null},{"id":"2606.06521","title":"P-Cast Precision in FP8 Attention: Sink-Induced Collapse and the Optimality of S=2^8","created":"2026-06-02","cited_by_count":0,"area":null},{"id":"2606.08105","title":"A Unifying View of Attention Sinks: From Mechanisms to Architectural Interventions","created":"2026-06-06","cited_by_count":0,"area":null},{"id":"2606.16112","title":"Scaling Adaptive Depth with Norm-Agnostic Residual Networks","created":"2026-06-15","cited_by_count":0,"area":null},{"id":"2606.17945","title":"Small Initialization Matters for Large Language Models","created":"2026-06-16","cited_by_count":0,"area":null},{"id":"2607.16824","title":"Test-Time Registers as Global Priors for Tokenized Image Generation","created":"2026-07-18","cited_by_count":0,"area":null},{"id":"2607.20484","title":"The Devil is in the Spectrum: Mitigating Representation Collapse in LLMs via Topologically Regularized Side-Path","created":"2026-06-02","cited_by_count":0,"area":null},{"id":"2607.20995","title":"Where Animacy Lives in Large Language Models: Tracing the Circuits of the Animacy Concept","created":"2026-07-23","cited_by_count":0,"area":null},{"id":"2608.02668","title":"Sphere Retraction Normalizations","created":"2026-08-02","cited_by_count":0,"area":null},{"id":"2608.09432","title":"ZetaGPT: A Reference Implementation of Positional--Encoding--Free State--Space--Attention Language Models","created":"2026-08-10","cited_by_count":0,"area":null},{"id":"2608.10296","title":"Cracks in the Foundation: Seemingly Minor Architectural Choices Impact Long Context Extension","created":"2026-08-10","cited_by_count":0,"area":null},{"id":"2608.21174","title":"From Attention Masks to Inert Zero-Vector Tokens: OAttention and O-Closure for Token Dynamics","created":"2026-08-21","cited_by_count":0,"area":null},{"id":"2609.08574","title":"Do New Attention Mechanisms Actually Fix Attention Sinks at Million-Token Context?","created":"2026-09-08","cited_by_count":0,"area":null},{"id":"2609.22005","title":"Abstention and Noise Filtering: Two Missing Primitives of Softmax Attention","created":"2026-09-18","cited_by_count":0,"area":null}],"next_cursor":"100"}