{"list":"other","total":66,"rows":[{"id":"2312.00752","title":"Mamba: Linear-Time Sequence Modeling with Selective State Spaces","created":"2023-12-01","cited_by_count":4718},{"id":"2205.14135","title":"FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness","created":"2022-05-27","cited_by_count":3673},{"id":"2004.05150","title":"Longformer: The Long-Document Transformer","created":"2020-04-10","cited_by_count":3413},{"id":"2307.08691","title":"FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning","created":"2023-07-17","cited_by_count":2598},{"id":"2111.00396","title":"Efficiently Modeling Long Sequences with Structured State Spaces","created":"2021-10-31","cited_by_count":2465},{"id":"1510.00149","title":"Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding","created":"2015-10-01","cited_by_count":2329},{"id":"1803.03635","title":"The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks","created":"2018-03-09","cited_by_count":2262},{"id":"2309.17453","title":"Efficient Streaming Language Models with Attention Sinks","created":"2023-09-29","cited_by_count":2186},{"id":"2210.17323","title":"GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers","created":"2022-10-31","cited_by_count":1919},{"id":"2007.14062","title":"Big Bird: Transformers for Longer Sequences","created":"2020-07-28","cited_by_count":1684},{"id":"2001.04451","title":"Reformer: The Efficient Transformer","created":"2020-01-13","cited_by_count":1668},{"id":"2009.14794","title":"Rethinking Attention with Performers","created":"2020-09-30","cited_by_count":1601},{"id":"1904.10509","title":"Generating Long Sequences with Sparse Transformers","created":"2019-04-23","cited_by_count":1546},{"id":"2006.04768","title":"Linformer: Self-Attention with Linear Complexity","created":"2020-06-08","cited_by_count":1474},{"id":"2301.00774","title":"SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot","created":"2023-01-02","cited_by_count":1121},{"id":"2108.12409","title":"Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation","created":"2021-08-27","cited_by_count":1103},{"id":"2209.11895","title":"In-context Learning and Induction Heads","created":"2022-09-24","cited_by_count":964},{"id":"2404.00456","title":"QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs","created":"2024-03-30","cited_by_count":634},{"id":"2003.05997","title":"Efficient Content-Based Sparse Attention with Routing Transformers","created":"2020-03-12","cited_by_count":471},{"id":"2405.16406","title":"SpinQuant: LLM quantization with learned rotations","created":"2024-05-26","cited_by_count":471},{"id":"2502.11089","title":"Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention","created":"2025-02-16","cited_by_count":407},{"id":"2302.10866","title":"Hyena Hierarchy: Towards Larger Convolutional Language Models","created":"2023-02-21","cited_by_count":393},{"id":"2401.15024","title":"SliceGPT: Compress Large Language Models by Deleting Rows and Columns","created":"2024-01-26","cited_by_count":385},{"id":"1404.0736","title":"Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation","created":"2014-04-02","cited_by_count":306},{"id":"2403.07378","title":"SVD-LLM: Truncation-aware Singular Value Decomposition for Large Language Model Compression","created":"2024-03-12","cited_by_count":243},{"id":"1810.01075","title":"Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learning","created":"2018-10-02","cited_by_count":215},{"id":"2502.13189","title":"MoBA: Mixture of Block Attention for Long-Context LLMs","created":"2025-02-18","cited_by_count":198},{"id":"2510.26692","title":"Kimi Linear: An Expressive, Efficient Attention Architecture","created":"2025-10-30","cited_by_count":196},{"id":"2207.00112","title":"Language model compression with weighted low-rank factorization","created":"2022-06-30","cited_by_count":195},{"id":"2312.05821","title":"ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Models","created":"2023-12-10","cited_by_count":169},{"id":"2310.05175","title":"Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity","created":"2023-10-08","cited_by_count":165},{"id":"1901.08276","title":"Traditional and Heavy-Tailed Self Regularization in Neural Network Models","created":"2019-01-24","cited_by_count":105},{"id":"1608.08225","title":"Why does deep and cheap learning work so well?","created":"2016-08-29","cited_by_count":100},{"id":"2506.19852","title":"Radial Attention: $O(n\\log n)$ Sparse Attention with Energy Decay for Long Video Generation","created":"2025-06-24","cited_by_count":69},{"id":"2008.08601","title":"Neural Networks and Quantum Field Theory","created":"2020-08-19","cited_by_count":61},{"id":"1410.3831","title":"An exact mapping between the Variational Renormalization Group and Deep Learning","created":"2014-10-14","cited_by_count":56},{"id":"1802.02840","title":"Neural Network Renormalization Group","created":"2018-02-08","cited_by_count":48},{"id":"1605.05775","title":"Supervised Learning with Quantum-Inspired Tensor Networks","created":"2016-05-18","cited_by_count":36},{"id":"1704.06279","title":"Mutual Information, Neural Networks and the Renormalization Group","created":"2017-04-20","cited_by_count":33},{"id":"2106.02205","title":"Enabling Lightweight Fine-tuning for Pre-trained Language Model Compression based on Matrix Product Operators","created":"2021-06-04","cited_by_count":29},{"id":"2504.17768","title":"The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs","created":"2025-04-24","cited_by_count":29},{"id":"1904.06194","title":"Compressing deep neural networks by matrix product operators","created":"2019-04-11","cited_by_count":28},{"id":"1901.10787","title":"Tensorized Embedding Layers for Efficient Model Compression","created":"2019-01-30","cited_by_count":27},{"id":"2108.01403","title":"Nonperturbative renormalization for the neural network-QFT correspondence","created":"2021-08-03","cited_by_count":24},{"id":"2410.17770","title":"Small Singular Values Matter: A Random Matrix Analysis of Transformer Models","created":"2024-10-23","cited_by_count":21},{"id":"2510.05554","title":"Critical attention scaling in long-context transformers","created":"2025-10-07","cited_by_count":20},{"id":"2307.00526","title":"TensorGPT: Efficient Compression of Large Language Models based on Tensor-Train Decomposition","created":"2023-07-02","cited_by_count":17},{"id":"2411.12892","title":"Selective Attention: Enhancing Transformer through Principled Context Control","created":"2024-11-19","cited_by_count":16},{"id":"2410.05437","title":"ESPACE: Dimensionality Reduction of Activations for Model Compression","created":"2024-10-07","cited_by_count":15},{"id":"2311.18807","title":"Pre-registration for Predictive Modeling","created":"2023-11-30","cited_by_count":13},{"id":"2501.16241","title":"Phase Transitions in Large Language Models and the $O(N)$ Model","created":"2025-01-27","cited_by_count":12},{"id":"2404.15729","title":"Gradformer: Graph Transformer with Exponential Decay","created":"2024-04-24","cited_by_count":11},{"id":"2405.15926","title":"Dissecting the Interplay of Attention Paths in a Statistical Mechanics Theory of Transformers","created":"2024-05-24","cited_by_count":9},{"id":"2110.03210","title":"Universality of Winning Tickets: A Renormalization Group Perspective","created":"2021-10-07","cited_by_count":8},{"id":"2507.01299","title":"La RoSA: Enhancing LLM Efficiency via Layerwise Rotated Sparse Activation","created":"2025-07-02","cited_by_count":6},{"id":"2511.20102","title":"SSA: Sparse Sparse Attention by Aligning Full and Sparse Attention Outputs in Feature Space","created":"2025-11-25","cited_by_count":6},{"id":"2404.16890","title":"Layer Collapse Can be Induced by Unstructured Pruning","created":"2024-04-24","cited_by_count":4},{"id":"2406.07247","title":"Dynamical Mean-Field Theory of Self-Attention Neural Networks","created":"2024-06-11","cited_by_count":4},{"id":"2606.03465","title":"Rethinking the Role of Tensor Decompositions in Post-Training LLM Compression","created":"2026-06-02","cited_by_count":4},{"id":"1609.03541","title":"Comment on \"Why does deep and cheap learning work so well?\" [arXiv:1608.08225]","created":"2016-09-12","cited_by_count":3},{"id":"2502.06151","title":"Recency Biased Causal Attention for Time-series Forecasting","created":"2025-02-10","cited_by_count":3},{"id":"2503.01922","title":"Pruning Deep Neural Networks via a Combination of the Marchenko-Pastur Distribution and Regularization","created":"2025-03-02","cited_by_count":3},{"id":"2603.28534","title":"Compressing Transformer Language Models via Matrix Product Operator Decomposition: A Case Study on PicoGPT","created":"2026-03-30","cited_by_count":3},{"id":"2506.05300","title":"Power Law Guided Dynamic Sifting for Efficient Attention","created":"2025-06-05","cited_by_count":1},{"id":"2607.20524","title":"Attention Degradation, Function Token Anchoring, and the Limits of Attention-Based Intervention in Large Language Models","created":"2026-07-09","cited_by_count":1},{"id":"2608.30505","title":"Tensor Methods for Language Models: From Token Representation to Training, Adaptation, Inference, Compression, and Interpretability","created":"2026-08-31","cited_by_count":1}],"next_cursor":null}