{"total":123,"rows":[{"id":"2402.05162","title":"Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications","created":"2024-02-07","cited_by_count":231,"area":"alignment"},{"id":"2403.17887","title":"The Unreasonable Ineffectiveness of the Deeper Layers","created":"2024-03-26","cited_by_count":226,"area":"lr"},{"id":"2402.16363","title":"LLM Inference Unveiled: Survey and Roofline Model Insights","created":"2024-02-26","cited_by_count":152,"area":"inference"},{"id":"2402.18815","title":"How do Large Language Models Handle Multilingualism?","created":"2024-02-29","cited_by_count":150,"area":null},{"id":"2406.19384","title":"The Remarkable Robustness of LLMs: Stages of Inference?","created":"2024-06-27","cited_by_count":146,"area":null},{"id":"2402.01761","title":"Rethinking Interpretability in the Era of Large Language Models","created":"2024-01-30","cited_by_count":111,"area":"lr"},{"id":"2410.21228","title":"LoRA vs Full Fine-tuning: An Illusion of Equivalence","created":"2024-10-28","cited_by_count":109,"area":null},{"id":"2406.09044","title":"MiLoRA: Harnessing Minor Singular Components for Parameter-Efficient LLM Finetuning","created":"2024-06-13","cited_by_count":94,"area":null},{"id":"2402.19450","title":"Functional Benchmarks for Robust Evaluation of Reasoning Performance, and the Reasoning Gap","created":"2024-02-29","cited_by_count":79,"area":"lr"},{"id":"2407.21118","title":"Palu: Compressing KV-Cache with Low-Rank Projection","created":"2024-07-30","cited_by_count":64,"area":"inference"},{"id":"2405.18886","title":"Compressing Large Language Models using Low Rank and Low Precision Decomposition","created":"2024-05-29","cited_by_count":59,"area":null},{"id":"2406.01563","title":"LoFiT: Localized Fine-tuning on LLM Representations","created":"2024-06-03","cited_by_count":57,"area":null},{"id":"2402.09221","title":"Spectral Filters, Dark Signals, and Attention Sinks","created":"2024-02-14","cited_by_count":56,"area":"mechinterp"},{"id":"2410.03111","title":"LoRC: Low-Rank Compression for LLMs KV Cache with a Progressive Compression Strategy","created":"2024-10-04","cited_by_count":51,"area":"inference"},{"id":"2406.02924","title":"Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for Large Language Models","created":"2024-06-05","cited_by_count":40,"area":null},{"id":"2410.23819","title":"Weight decay induces low-rank attention layers","created":"2024-10-31","cited_by_count":27,"area":null},{"id":"2401.10862","title":"Pruning for Protection: Increasing Jailbreak Resistance in Aligned LLMs Without Fine-Tuning","created":"2024-01-19","cited_by_count":26,"area":"safety"},{"id":"2404.18923","title":"Holmes: A Benchmark to Assess the Linguistic Competence of Language Models","created":"2024-04-29","cited_by_count":26,"area":null},{"id":"2407.00066","title":"Compress then Serve: Serving Thousands of LoRA Adapters with Little Overhead","created":"2024-06-17","cited_by_count":24,"area":null},{"id":"2512.05117","title":"The Universal Weight Subspace Hypothesis","created":"2025-12-04","cited_by_count":24,"area":null},{"id":"2403.19243","title":"Efficient Learning With Sine-Activated Low-rank Matrices","created":"2024-03-28","cited_by_count":22,"area":null},{"id":"2402.01799","title":"Faster and Lighter LLMs: A Survey on Current Challenges and Way Forward","created":"2024-02-02","cited_by_count":21,"area":null},{"id":"2410.17770","title":"Small Singular Values Matter: A Random Matrix Analysis of Transformer Models","created":"2024-10-23","cited_by_count":21,"area":null},{"id":"2410.04960","title":"On Efficient Variants of Segment Anything Model: A Survey","created":"2024-10-07","cited_by_count":16,"area":null},{"id":"2403.09053","title":"Towards a theory of model distillation","created":"2024-03-14","cited_by_count":15,"area":null},{"id":"2405.15302","title":"Understanding the Language Model to Solve the Symbolic Multi-Step Reasoning Problem from the Perspective of Buffer Mechanism","created":"2024-05-24","cited_by_count":15,"area":"lr"},{"id":"2410.03090","title":"UNComp: Can Matrix Entropy Uncover Sparsity? -- A Compressor Design from an Uncertainty-Aware Perspective","created":"2024-10-04","cited_by_count":15,"area":"inference"},{"id":"2504.07097","title":"Sculpting Subspaces: Constrained Full Fine-Tuning in LLMs for Continual Learning","created":"2025-04-09","cited_by_count":15,"area":null},{"id":"2407.10886","title":"SLIP: Securing LLMs IP Using Weights Decomposition","created":"2024-07-15","cited_by_count":14,"area":null},{"id":"2504.09923","title":"Guiding Reasoning in Small Language Models with LLM Assistance","created":"2025-04-14","cited_by_count":14,"area":"lr"},{"id":"2408.10774","title":"Flexora: Flexible Low Rank Adaptation for Large Language Models","created":"2024-08-20","cited_by_count":13,"area":null},{"id":"2506.14562","title":"AlphaDecay: Module-wise Weight Decay for Heavy-Tailed Balancing in LLMs","created":"2025-06-17","cited_by_count":13,"area":null},{"id":"2406.03068","title":"Distributional Associations vs In-Context Reasoning: A Study of Feed-forward and Attention Layers","created":"2024-06-05","cited_by_count":12,"area":"lr"},{"id":"2501.15674","title":"TensorLLM: Tensorising Multi-Head Attention for Enhanced Reasoning and Compression in LLMs","created":"2025-01-26","cited_by_count":12,"area":"lr"},{"id":"2503.06881","title":"ResMoE: Space-efficient Compression of Mixture of Experts LLMs via Residual Restoration","created":"2025-03-10","cited_by_count":12,"area":"moe"},{"id":"2603.10067","title":"HTMuon: Improving Muon via Heavy-Tailed Spectral Correction","created":"2026-03-10","cited_by_count":12,"area":null},{"id":"2602.00135","title":"LLaVA-FA: Learning Fourier Approximation for Compressing Large Multimodal Models","created":"2026-01-28","cited_by_count":11,"area":"mllm"},{"id":"2405.10616","title":"Adaptive Feature-based Low-Rank Compression of Large Language Models via Bayesian Optimization","created":"2024-05-17","cited_by_count":10,"area":null},{"id":"2406.12629","title":"SeTAR: Out-of-Distribution Detection with Selective Low-Rank Approximation","created":"2024-06-18","cited_by_count":10,"area":null},{"id":"2412.08258","title":"Large Language Models for Scholarly Ontology Generation: An Extensive Analysis in the Engineering Field","created":"2024-12-11","cited_by_count":10,"area":null},{"id":"2505.18752","title":"Unifying Attention Heads and Task Vectors via Hidden State Geometry in In-Context Learning","created":"2025-05-24","cited_by_count":10,"area":null},{"id":"2405.15007","title":"RE-Adapt: Reverse Engineered Adaptation of Large Language Models","created":"2024-05-23","cited_by_count":9,"area":null},{"id":"2406.15972","title":"EVCL: Elastic Variational Continual Learning with Weight Consolidation","created":"2024-06-23","cited_by_count":9,"area":null},{"id":"2410.16801","title":"Controlled Low-Rank Adaptation with Subspace Regularization for Continued Training on Large Language Models","created":"2024-10-22","cited_by_count":9,"area":null},{"id":"2502.04700","title":"EigenLoRAx: Recycling Adapters to Find Principal Subspaces for Resource-Efficient Adaptation and Inference","created":"2025-02-07","cited_by_count":9,"area":null},{"id":"2510.03358","title":"Understanding Transformers for Time Series: Rank Structure, Flow-of-ranks, and Compressibility","created":"2025-10-02","cited_by_count":9,"area":null},{"id":"2511.00280","title":"Calibration Across Layers: Understanding Calibration Evolution in LLMs","created":"2025-10-31","cited_by_count":9,"area":null},{"id":"2412.06071","title":"KaSA: Knowledge-Aware Singular-Value Adaptation of Large Language Models","created":"2024-12-08","cited_by_count":8,"area":null},{"id":"2406.17887","title":"Federated Dynamical Low-Rank Training with Global Loss Convergence Guarantees","created":"2024-06-25","cited_by_count":7,"area":null},{"id":"2407.09835","title":"Investigating Low-Rank Training in Transformer Language Models: Efficiency and Scaling Analysis","created":"2024-07-13","cited_by_count":6,"area":null},{"id":"2410.22517","title":"Attention Speaks Volumes: Localizing and Mitigating Bias in Language Models","created":"2024-10-29","cited_by_count":6,"area":null},{"id":"2602.12429","title":"Stabilizing Native Low-Rank LLM Pretraining","created":"2026-02-12","cited_by_count":6,"area":null},{"id":"2604.19974","title":"Are LLM Uncertainty and Correctness Encoded by the Same Features? A Functional Dissociation via Sparse Autoencoders","created":"2026-04-21","cited_by_count":6,"area":"mechinterp"},{"id":"2404.15198","title":"Lossless and Near-Lossless Compression for Foundation Models","created":"2024-04-05","cited_by_count":5,"area":null},{"id":"2406.17261","title":"TRAWL: Tensor Reduced and Approximated Weights for Large Language Models","created":"2024-06-25","cited_by_count":5,"area":null},{"id":"2407.16153","title":"On the Benefits of Rank in Attention Layers","created":"2024-07-23","cited_by_count":5,"area":null},{"id":"2504.13237","title":"ImPart: Importance-Aware Delta-Sparsification for Improved Model Compression and Merging in LLMs","created":"2025-04-17","cited_by_count":5,"area":null},{"id":"2406.03768","title":"Enhancing In-Context Learning Performance with just SVD-Based Weight Pruning: A Theoretical Perspective","created":"2024-06-06","cited_by_count":4,"area":null},{"id":"2407.04797","title":"Revealing the Utilized Rank of Subspaces of Learning in Neural Networks","created":"2024-07-05","cited_by_count":4,"area":null},{"id":"2407.06483","title":"Composable Interventions for Language Models","created":"2024-07-09","cited_by_count":4,"area":"safety"},{"id":"2409.03856","title":"Sirius: Contextual Sparsity with Correction for Efficient LLMs","created":"2024-09-05","cited_by_count":4,"area":null},{"id":"2410.00340","title":"Sparse Attention Decomposition Applied to Circuit Tracing","created":"2024-10-01","cited_by_count":4,"area":null},{"id":"2410.02147","title":"Efficient Source-Free Time-Series Adaptation via Parameter Subspace Disentanglement","created":"2024-10-03","cited_by_count":4,"area":null},{"id":"2501.19090","title":"Pivoting Factorization: A Compact Meta Low-Rank Representation of Sparsity for Efficient Inference in Large Language Models","created":"2025-01-31","cited_by_count":4,"area":null},{"id":"2606.03465","title":"Rethinking the Role of Tensor Decompositions in Post-Training LLM Compression","created":"2026-06-02","cited_by_count":4,"area":"lr"},{"id":"2402.16319","title":"Data-free Weight Compress and Denoise for Large Language Models","created":"2024-02-26","cited_by_count":3,"area":null},{"id":"2406.06623","title":"Spectrum: Targeted Training on Signal to Noise Ratio","created":"2024-06-07","cited_by_count":3,"area":null},{"id":"2502.01717","title":"Choose Your Model Size: Any Compression of Large Language Models Without Re-Computation","created":"2025-02-03","cited_by_count":3,"area":null},{"id":"2502.20681","title":"Disentangling Feature Structure: A Mathematically Provable Two-Stage Training Dynamics in Transformers","created":"2025-02-28","cited_by_count":3,"area":null},{"id":"2505.18350","title":"How Many Parameters Does Your Task Really Need? Task Specific Pruning with LLM-Sieve","created":"2025-05-23","cited_by_count":3,"area":null},{"id":"2505.18575","title":"Response Uncertainty and Probe Modeling: Two Sides of the Same Coin in LLM Interpretability?","created":"2025-05-24","cited_by_count":3,"area":null},{"id":"2506.02818","title":"ProcrustesGPT: Compressing LLMs with Structured Matrices and Orthogonal Transformations","created":"2025-06-03","cited_by_count":3,"area":null},{"id":"2601.05478","title":"The Facade of Truth: Uncovering and Mitigating LLM Susceptibility to Deceptive Evidence","created":"2026-01-09","cited_by_count":3,"area":null},{"id":"2601.19723","title":"Component-Level Lesioning of Language Models Reveals Clinically Aligned Aphasia Phenotypes","created":"2026-01-27","cited_by_count":3,"area":null},{"id":"2603.03995","title":"Spectral Surgery: Training-Free Refinement of LoRA via Gradient-Guided Singular Value Reweighting","created":"2026-03-04","cited_by_count":3,"area":null},{"id":"2604.15351","title":"Aletheia: Gradient-Guided Layer Selection for Efficient LoRA Fine-Tuning Across Architectures","created":"2026-04-04","cited_by_count":3,"area":null},{"id":"2605.20296","title":"Spectral Unforgetting: Post-Hoc Recovery of Damaged Capabilities Without Retraining","created":"2026-05-19","cited_by_count":3,"area":null},{"id":"2406.16450","title":"Building on Efficient Foundations: Effectively Training LLMs with Structured Feedforward Layers","created":"2024-06-24","cited_by_count":2,"area":null},{"id":"2407.00356","title":"Enhancing Accuracy and Parameter-Efficiency of Neural Representations for Network Parameterization","created":"2024-06-29","cited_by_count":2,"area":null},{"id":"2407.14655","title":"LORTSAR: Low-Rank Transformer for Skeleton-based Action Recognition","created":"2024-07-19","cited_by_count":2,"area":null},{"id":"2510.24256","title":"From Memorization to Reasoning in the Spectrum of Loss Curvature","created":"2025-10-28","cited_by_count":2,"area":"lr"},{"id":"2512.22671","title":"Fragile Knowledge, Robust Instruction-Following: The Width Pruning Dichotomy in Llama-3.2","created":"2025-12-27","cited_by_count":2,"area":null},{"id":"2602.06043","title":"Shared LoRA Subspaces for almost Strict Continual Learning","created":"2026-02-05","cited_by_count":2,"area":null},{"id":"2606.23670","title":"Tapered Language Models","created":"2026-06-22","cited_by_count":2,"area":null},{"id":"2401.15713","title":"Contrastive Learning and Mixture of Experts Enables Precise Vector Embeddings","created":"2024-01-28","cited_by_count":1,"area":"moe"},{"id":"2405.15877","title":"Basis Selection: Low-Rank Decomposition of Pretrained Large Language Models for Target Applications","created":"2024-05-24","cited_by_count":1,"area":null},{"id":"2407.07802","title":"ROSA: Random Subspace Adaptation for Efficient Fine-Tuning","created":"2024-07-10","cited_by_count":1,"area":null},{"id":"2501.04211","title":"CURing Large Models: Compression via CUR Decomposition","created":"2025-01-08","cited_by_count":1,"area":null},{"id":"2505.15090","title":"DeFTX: Denoised Sparse Fine-Tuning for Zero-Shot Cross-Lingual Transfer","created":"2025-05-21","cited_by_count":1,"area":null},{"id":"2506.02561","title":"Pruning General Large Language Models into Customized Expert Models","created":"2025-06-03","cited_by_count":1,"area":null},{"id":"2507.03828","title":"IMPACT: Importance-Aware Activation Space Reconstruction","created":"2025-07-04","cited_by_count":1,"area":null},{"id":"2510.20800","title":"Compress to Impress: Efficient LLM Adaptation Using a Single Gradient Step on 100 Samples","created":"2025-10-23","cited_by_count":1,"area":null},{"id":"2511.19518","title":"Towards Efficient VLMs: Information-Theoretic Driven Compression via Adaptive Structural Pruning","created":"2025-11-24","cited_by_count":1,"area":"mllm"},{"id":"2512.08082","title":"Short-Context Dominance: How Much Local Context Natural Language Actually Needs?","created":"2025-12-08","cited_by_count":1,"area":null},{"id":"2601.17112","title":"Low-Rank Tensor Approximation of Weights in Large Language Models via Cosine Lanczos Bidiagonalization","created":"2026-01-23","cited_by_count":1,"area":null},{"id":"2602.09066","title":"Spectral Disentanglement and Enhancement: A Dual-domain Contrastive Framework for Representation Learning","created":"2026-02-09","cited_by_count":1,"area":null},{"id":"2602.22247","title":"Multi-Dimensional Spectral Geometry of Biological Knowledge in Single-Cell Transformer Representations","created":"2026-02-24","cited_by_count":1,"area":null},{"id":"2603.10985","title":"The Discrete Charm of the MLP: Binary Routing of Continuous Signals in Transformer Feed-Forward Layers","created":"2026-03-11","cited_by_count":1,"area":null},{"id":"2604.19351","title":"DASH-KV: Accelerating Long-Context LLM Inference via Asymmetric KV Cache Hashing","created":"2026-04-21","cited_by_count":1,"area":"inference"},{"id":"2605.16600","title":"Where Pretraining writes and Alignment reads: the asymmetry of Transformer weight space","created":"2026-05-15","cited_by_count":1,"area":"alignment"}],"next_cursor":"100"}