{"total":90,"rows":[{"id":"2502.05564","title":"TabICL: A Tabular Foundation Model for In-Context Learning on Large Data","created":"2025-02-08","cited_by_count":289,"area":null},{"id":"2504.02732","title":"Why do LLMs attend to the first token?","created":"2025-04-03","cited_by_count":127,"area":null},{"id":"2502.05173","title":"VideoRoPE: What Makes for Good Video Rotary Position Embedding?","created":"2025-02-07","cited_by_count":65,"area":null},{"id":"2502.01951","title":"On the Emergence of Position Bias in Transformers","created":"2025-02-04","cited_by_count":59,"area":null},{"id":"2510.06477","title":"Attention Sinks and Compression Valleys in LLMs are Two Sides of the Same Coin","created":"2025-10-07","cited_by_count":52,"area":null},{"id":"2501.18795","title":"Rope to Nope and Back Again: A New Hybrid Attention Strategy","created":"2025-01-30","cited_by_count":47,"area":null},{"id":"2502.01563","title":"Massive Values in Self-Attention Modules are the Key to Contextual Knowledge Understanding","created":"2025-02-03","cited_by_count":47,"area":null},{"id":"2412.17739","title":"Fourier Position Embedding: Enhancing Attention's Periodic Extension for Length Generalization","created":"2024-12-23","cited_by_count":39,"area":null},{"id":"2505.24722","title":"HELM: Hyperbolic Large Language Models via Mixture-of-Curvature Experts","created":"2025-05-30","cited_by_count":26,"area":null},{"id":"2512.12167","title":"Extending the Context of Pretrained LLMs by Dropping Their Positional Embeddings","created":"2025-12-13","cited_by_count":26,"area":null},{"id":"2506.16640","title":"Long-Context Generalization with Sparse Attention","created":"2025-06-19","cited_by_count":21,"area":null},{"id":"2502.02562","title":"Learning the RoPEs: Better 2D and 3D Position Encodings with STRING","created":"2025-02-04","cited_by_count":19,"area":null},{"id":"2502.00919","title":"Attention Sinks: A 'Catch, Tag, Release' Mechanism for Embeddings","created":"2025-02-02","cited_by_count":16,"area":null},{"id":"2503.08908","title":"Interpreting the Repeated Token Phenomenon in Large Language Models","created":"2025-03-11","cited_by_count":16,"area":null},{"id":"2505.13755","title":"Panda: A pretrained forecast model for chaotic dynamics","created":"2025-05-19","cited_by_count":13,"area":null},{"id":"2506.09251","title":"Extrapolation by Association: Length Generalization Transfer in Transformers","created":"2025-06-10","cited_by_count":13,"area":null},{"id":"2511.19827","title":"ReDirector: Creating Any-Length Video Retakes with Rotary Camera Encoding","created":"2025-11-25","cited_by_count":11,"area":null},{"id":"2509.05218","title":"HoPE: Hyperbolic Rotary Positional Encoding for Stable Long-Range Dependency Modeling in Large Language Models","created":"2025-09-05","cited_by_count":10,"area":null},{"id":"2509.10534","title":"Decoupling the \"What\" and \"Where\" With Polar Coordinate Positional Embeddings","created":"2025-09-05","cited_by_count":10,"area":null},{"id":"2605.15514","title":"RoPE Distinguishes Neither Positions Nor Tokens in Long Contexts, Provably","created":"2026-05-15","cited_by_count":8,"area":null},{"id":"2508.02215","title":"LeanK: Learnable K Cache Channel Pruning for Efficient Decoding","created":"2025-08-04","cited_by_count":7,"area":null},{"id":"2510.26784","title":"LLMs Process Lists With General Filter Heads","created":"2025-10-30","cited_by_count":7,"area":null},{"id":"2511.11579","title":"Decoupling Positional and Symbolic Attention Behavior in Transformers","created":"2025-10-03","cited_by_count":6,"area":null},{"id":"2505.13027","title":"Deconstructing Positional Information: From Attention Logits to Training Biases","created":"2025-05-19","cited_by_count":5,"area":null},{"id":"2505.17083","title":"Scale-invariant Attention","created":"2025-05-20","cited_by_count":5,"area":null},{"id":"2512.07805","title":"Group Representational Position Encoding","created":"2025-12-08","cited_by_count":5,"area":null},{"id":"2512.14391","title":"RePo: Language Models with Context Re-Positioning","created":"2025-12-16","cited_by_count":4,"area":null},{"id":"2601.22181","title":"MrRoPE: Mixed-radix Rotary Position Embedding","created":"2026-01-28","cited_by_count":4,"area":null},{"id":"2603.11611","title":"Fractional Rotation, Full Potential? Investigating Performance and Convergence of Partial RoPE","created":"2026-03-12","cited_by_count":4,"area":null},{"id":"2511.08368","title":"A Circular Argument : Does RoPE need to be Equivariant for Vision?","created":"2025-11-11","cited_by_count":3,"area":null},{"id":"2602.04752","title":"Decomposing Query-Key Feature Interactions Using Contrastive Covariances","created":"2026-02-04","cited_by_count":3,"area":null},{"id":"2603.00563","title":"Whisper-MLA: Reducing GPU Memory Consumption of ASR Models based on MHA2MLA Conversion","created":"2026-02-28","cited_by_count":3,"area":null},{"id":"2603.18017","title":"Frayed RoPE and Long Inputs: A Geometric Perspective","created":"2026-02-24","cited_by_count":3,"area":null},{"id":"2605.12922","title":"When Attention Closes: How LLMs Lose the Thread in Multi-Turn Interaction","created":"2026-05-13","cited_by_count":3,"area":null},{"id":"2609.02986","title":"Modern Transformers Are Implicit Hybrids: From Functional Differentiation to Principled Hybrid Architecture Design","created":"2026-09-02","cited_by_count":3,"area":null},{"id":"2410.18067","title":"Beyond Position: the emergence of wavelet-like properties in Transformers","created":"2024-10-23","cited_by_count":2,"area":null},{"id":"2503.08067","title":"Context-aware Biases for Length Extrapolation","created":"2025-03-11","cited_by_count":2,"area":null},{"id":"2509.11208","title":"Predictable Compression Failures: Order Sensitivity and Information Budgeting for Evidence-Grounded Binary Adjudication","created":"2025-09-14","cited_by_count":2,"area":null},{"id":"2512.07525","title":"Beyond Real: Imaginary Extension of Rotary Position Embeddings for Long-Context LLMs","created":"2025-12-08","cited_by_count":2,"area":null},{"id":"2601.12083","title":"Learning to Factorize and Adapt: A Versatile Approach Toward Universal Spatio-Temporal Foundation Models","created":"2026-01-17","cited_by_count":2,"area":null},{"id":"2602.05258","title":"CoPE: Clipped RoPE as A Scalable Free Lunch for Long Context LLMs","created":"2026-02-05","cited_by_count":2,"area":null},{"id":"2604.09527","title":"Envisioning the Future, One Step at a Time","created":"2026-04-10","cited_by_count":2,"area":null},{"id":"2605.00968","title":"Adaptive 3D-RoPE: Physics-Aligned Rotary Positional Encoding for Wireless Foundation Models","created":"2026-05-01","cited_by_count":2,"area":null},{"id":"2606.11275","title":"RoVE: Rotary Value Embeddings Attention for Relative Position-dependent Value Pathways","created":"2026-06-09","cited_by_count":2,"area":null},{"id":"2607.07678","title":"How Data Shapes RoPE Frequency Usage: From Positional Scale Matching to Length Generalization","created":"2026-07-08","cited_by_count":2,"area":null},{"id":"2503.01832","title":"Rotary Offset Features in Large Language Models","created":"2025-03-03","cited_by_count":1,"area":null},{"id":"2503.23174","title":"TRA: Better Length Generalisation with Threshold Relative Attention","created":"2025-03-29","cited_by_count":1,"area":null},{"id":"2509.21042","title":"LayerNorm Induces Recency Bias in Transformer Decoders","created":"2025-09-25","cited_by_count":1,"area":null},{"id":"2510.10489","title":"Head-wise Adaptive Rotary Positional Encoding for Fine-Grained Image Generation","created":"2025-10-12","cited_by_count":1,"area":null},{"id":"2511.22316","title":"Outlier Smoothing with Closed-Form Rotations for W4A4 Large Language Model Quantization","created":"2025-11-27","cited_by_count":1,"area":null},{"id":"2604.11737","title":"Learning Long-term Motion Embeddings for Efficient Kinematics Generation","created":"2026-04-13","cited_by_count":1,"area":null},{"id":"2605.10414","title":"Remember to Forget: Gated Adaptive Positional Encoding","created":"2026-05-11","cited_by_count":1,"area":null},{"id":"2605.31558","title":"Positional versus Symbolic Attention Heads: Learning Dynamics, RoPE Geometry, and Length Generalization","created":"2026-05-29","cited_by_count":1,"area":null},{"id":"2607.10134","title":"LeRoPE: Learnable RoPE Frequencies Improve Language Modeling","created":"2026-07-11","cited_by_count":1,"area":null},{"id":"2607.13458","title":"2D Rotary Position Embedding for Scene Text Recognition with Transformers","created":"2026-07-15","cited_by_count":1,"area":null},{"id":"2607.19363","title":"AdaRoPE: Not All Attention Heads Should Rotate and Scale Equally","created":"2026-06-05","cited_by_count":1,"area":null},{"id":"2607.25507","title":"Phase Structure in Rotary Attention: A Spectral Framework for Semantic Continuity and Execution-Boundary Governance","created":"2026-07-28","cited_by_count":1,"area":null},{"id":"2608.11909","title":"Disentangling the Expressivity of RoPE","created":"2026-08-12","cited_by_count":1,"area":null},{"id":"2503.04377","title":"How can representation dimension dominate structurally pruned LLMs?","created":"2025-03-06","cited_by_count":0,"area":null},{"id":"2504.04238","title":"Sensitivity Meets Sparsity: The Impact of Extremely Sparse Parameter Patterns on Theory-of-Mind of Large Language Models","created":"2025-04-05","cited_by_count":0,"area":null},{"id":"2504.05364","title":"Of All StrIPEs: Investigating Structure-informed Positional Encoding for Efficient Music Generation","created":"2025-04-07","cited_by_count":0,"area":null},{"id":"2504.18857","title":"Effective Length Extrapolation via Dimension-Wise Positional Embeddings Manipulation","created":"2025-04-26","cited_by_count":0,"area":null},{"id":"2509.12196","title":"Dynamic Relational Priming Improves Transformer in Multivariate Time Series","created":"2025-09-15","cited_by_count":0,"area":null},{"id":"2509.23178","title":"Limit Analysis for Symbolic Multi-step Reasoning Tasks with Information Propagation Rules Based on Transformers","created":"2025-09-27","cited_by_count":0,"area":null},{"id":"2511.11668","title":"Do traveling waves make good positional encodings?","created":"2025-11-11","cited_by_count":0,"area":null},{"id":"2511.21733","title":"RoSA: Enhancing Parameter-Efficient Fine-Tuning via RoPE-aware Selective Adaptation in Large Language Models","created":"2025-11-21","cited_by_count":0,"area":null},{"id":"2512.05385","title":"ShaRP: SHAllow-LayeR Pruning for Efficient Video Large Language Models","created":"2025-12-05","cited_by_count":0,"area":null},{"id":"2512.15600","title":"How Smoothing is N-simplicial Attention?","created":"2025-12-17","cited_by_count":0,"area":null},{"id":"2601.14522","title":"On the Runway Cascade of Transformers for Language Modeling","created":"2026-01-20","cited_by_count":0,"area":null},{"id":"2602.01418","title":"Parabolic Position Encoding: Vision-Centric, Principled, Extrapolatable, General","created":"2026-02-01","cited_by_count":0,"area":null},{"id":"2602.01845","title":"No Generation without Representation: Efficient Causal Protein Language Models Enable Zero-Shot Fitness Estimation","created":"2026-02-02","cited_by_count":0,"area":null},{"id":"2602.06534","title":"AlertBERT: A noise-robust alert grouping framework for simultaneous cyber attacks","created":"2026-02-06","cited_by_count":0,"area":null},{"id":"2604.18603","title":"Dual Triangle Attention: Effective Bidirectional Attention Without Positional Embeddings","created":"2026-04-09","cited_by_count":0,"area":null},{"id":"2606.12213","title":"SHERPA: Seam-aware Harmonized ERP Adaptation for Open-Domain 360$^\\circ$ Panorama Generation","created":"2026-06-10","cited_by_count":0,"area":null},{"id":"2606.21249","title":"Does RoPE Prevent or Degrade Retrieval Heads? A Mechanistic Analysis Across Model Families","created":"2026-06-19","cited_by_count":0,"area":null},{"id":"2607.06621","title":"Fingerprint, Not Blueprint: How Positional Schemes Set the Default Spectral Algebra of Attention","created":"2026-07-07","cited_by_count":0,"area":null},{"id":"2607.18759","title":"Relative Positions Generalize, Absolute Positions Memorize: An Implicit-Bias Account of Length Generalization in Attention","created":"2026-07-21","cited_by_count":0,"area":null},{"id":"2607.23194","title":"Out-of-Length Scene Text Recognition: A Two-Axis Diagnosis and a Training-Free Fix","created":"2026-07-25","cited_by_count":0,"area":null},{"id":"2608.03994","title":"When Attention Goes Blind: Numerical Failure in ALiBi Positional Encodings","created":"2026-08-04","cited_by_count":0,"area":null},{"id":"2609.14320","title":"SpectralShift: Effective Context Window Extension of Gated DeltaNet via Spectral Reparameterization","created":"2026-09-13","cited_by_count":0,"area":null},{"id":"2609.31261","title":"MoSAR: Mixture of Semantic Attention Regimes for Learning Adaptive and Approximable Attention Geometries","created":"2026-09-25","cited_by_count":0,"area":null},{"id":"2609.34556","title":"RoPE is Dead, Long Live RoPE: Towards Scalable Data-aware Positional Encodings","created":"2026-09-28","cited_by_count":0,"area":null},{"id":"2609.34650","title":"When Can Attention Heads Be Statically Defined?","created":"2026-09-28","cited_by_count":0,"area":null},{"id":"2609.35852","title":"A Mesoscopic View of Transformer Weights Through Row and Column Scale Fields","created":"2026-09-25","cited_by_count":0,"area":null},{"id":"2609.38109","title":"How Local Mixing Encodes Relative Position in Global NoPE Attention","created":"2026-09-29","cited_by_count":0,"area":null},{"id":"2609.38530","title":"Shifting Mechanisms: How Positional Encoding Choice Shapes In-Context Retrieval","created":"2026-09-29","cited_by_count":0,"area":null},{"id":"2610.04791","title":"Not All Answers Are Contextually Persuadable: Inference Dynamics in Large Language Models under Contextual Influence","created":"2026-10-03","cited_by_count":0,"area":null},{"id":"2610.06963","title":"WavePrune: One period is often enough for RoPE","created":"2026-10-03","cited_by_count":0,"area":null},{"id":"2610.07848","title":"Dynamic Positional Attention Modulation for Parameter-Efficient Fine-Tuning of Large Language Models","created":"2026-10-06","cited_by_count":0,"area":null},{"id":"2610.13992","title":"How Vision Transformers Represent Position: A Mechanistic Comparison of Absolute and Rotary Positional Encodings","created":"2026-09-26","cited_by_count":0,"area":null}],"next_cursor":null}