{"total":107,"rows":[{"id":"2106.10199","title":"BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models","created":"2021-06-18","cited_by_count":1245,"area":null},{"id":"2111.01243","title":"Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey","created":"2021-11-01","cited_by_count":463,"area":null},{"id":"2106.04647","title":"Compacter: Efficient Low-Rank Hypercomplex Adapter Layers","created":"2021-06-08","cited_by_count":460,"area":null},{"id":"2106.07139","title":"Pre-Trained Models: Past, Present and Future","created":"2021-06-14","cited_by_count":319,"area":null},{"id":"2308.06767","title":"A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendations","created":"2023-08-13","cited_by_count":202,"area":null},{"id":"2204.00408","title":"Structured Pruning Learns Compact and Accurate Models","created":"2022-04-01","cited_by_count":170,"area":null},{"id":"2204.09656","title":"A Fast Post-Training Pruning Framework for Transformers","created":"2022-03-29","cited_by_count":162,"area":null},{"id":"2110.07560","title":"Composable Sparse Fine-Tuning for Cross-Lingual Transfer","created":"2021-10-14","cited_by_count":144,"area":null},{"id":"2109.12948","title":"Understanding and Overcoming the Challenges of Efficient Transformer Quantization","created":"2021-09-27","cited_by_count":141,"area":null},{"id":"2105.06990","title":"BERT Busters: Outlier Dimensions that Disrupt Transformers","created":"2021-05-14","cited_by_count":112,"area":null},{"id":"2302.06600","title":"Task-Specific Skill Localization in Fine-tuned Language Models","created":"2023-02-13","cited_by_count":105,"area":null},{"id":"2302.11529","title":"Modular Deep Learning","created":"2023-02-22","cited_by_count":83,"area":null},{"id":"2103.14636","title":"A Practical Survey on Faster and Lighter Transformers","created":"2021-03-26","cited_by_count":73,"area":null},{"id":"2402.09748","title":"Model Compression and Efficient Inference for Large Language Models: A Survey","created":"2024-02-15","cited_by_count":64,"area":null},{"id":"2105.12002","title":"Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization","created":"2021-05-25","cited_by_count":56,"area":null},{"id":"2205.11380","title":"Outliers Dimensions that Disrupt Transformers Are Driven by Frequency","created":"2022-05-23","cited_by_count":55,"area":null},{"id":"2106.05933","title":"PARP: Prune, Adjust and Re-Prune for Self-Supervised Speech Recognition","created":"2021-06-10","cited_by_count":53,"area":null},{"id":"2107.00166","title":"Sanity Checks for Lottery Tickets: Does Your Winning Ticket Really Win the Jackpot?","created":"2021-07-01","cited_by_count":52,"area":null},{"id":"2111.05193","title":"A Survey on Green Deep Learning","created":"2021-11-08","cited_by_count":52,"area":null},{"id":"2108.04657","title":"Differentiable Subset Pruning of Transformer Heads","created":"2021-08-10","cited_by_count":47,"area":null},{"id":"2104.11832","title":"Playing Lottery Tickets with Vision and Language","created":"2021-04-23","cited_by_count":45,"area":null},{"id":"2202.07105","title":"A Survey on Model Compression and Acceleration for Pretrained Language Models","created":"2022-02-15","cited_by_count":45,"area":null},{"id":"2306.03805","title":"The Emergence of Essential Sparsity in Large Pre-trained Models: The Weights that Matter","created":"2023-06-06","cited_by_count":42,"area":null},{"id":"2312.10136","title":"Gradient-based Parameter Selection for Efficient Fine-Tuning","created":"2023-12-15","cited_by_count":40,"area":null},{"id":"2312.01429","title":"Transformers are uninterpretable with myopic methods: a case study with bounded Dyck grammars","created":"2023-12-03","cited_by_count":30,"area":null},{"id":"2110.14068","title":"Drawing Robust Scratch Tickets: Subnetworks with Inborn Robustness Are Found within Randomly Initialized Networks","created":"2021-10-26","cited_by_count":29,"area":null},{"id":"2205.12672","title":"Discovering Language-neutral Sub-networks in Multilingual Language Models","created":"2022-05-25","cited_by_count":28,"area":null},{"id":"2109.09920","title":"Survey: Transformer based Video-Language Pre-training","created":"2021-09-21","cited_by_count":27,"area":null},{"id":"2110.10030","title":"Accelerating Framework of Transformer by Hardware Design and Model Compression Co-Optimization","created":"2021-10-19","cited_by_count":27,"area":null},{"id":"2303.04185","title":"Gradient-Free Structured Pruning with Unlabeled Data","created":"2023-03-07","cited_by_count":26,"area":null},{"id":"2111.11802","title":"Pruning Self-attentions into Convolutional Layers in Single Path","created":"2021-11-23","cited_by_count":24,"area":null},{"id":"2403.04861","title":"A Survey of Lottery Ticket Hypothesis","created":"2024-03-07","cited_by_count":24,"area":null},{"id":"2305.02190","title":"Rethinking Graph Lottery Tickets: Graph Sparsity Matters","created":"2023-05-03","cited_by_count":23,"area":null},{"id":"2312.15398","title":"Fairness-Aware Structured Pruning in Transformers","created":"2023-12-24","cited_by_count":22,"area":null},{"id":"2206.13288","title":"Discovering Salient Neurons in Deep NLP Models","created":"2022-06-27","cited_by_count":21,"area":null},{"id":"2201.08542","title":"Can Model Compression Improve NLP Fairness","created":"2022-01-21","cited_by_count":20,"area":null},{"id":"2306.10460","title":"Instant Soup: Cheap Pruning Ensembles in A Single Pass Can Draw Lottery Tickets from Large Models","created":"2023-06-18","cited_by_count":20,"area":null},{"id":"2112.07198","title":"From Dense to Sparse: Contrastive Pruning for Better Pre-trained Language Model Compression","created":"2021-12-14","cited_by_count":19,"area":null},{"id":"2305.17446","title":"Fine-tuning Happens in Tiny Subspaces: Exploring Intrinsic Task-specific Subspaces of Pre-trained Language Models","created":"2023-05-27","cited_by_count":19,"area":null},{"id":"2211.03013","title":"Robust Lottery Tickets for Pre-trained Language Models","created":"2022-11-06","cited_by_count":16,"area":null},{"id":"2310.17715","title":"Outlier Dimensions Encode Task-Specific Knowledge","created":"2023-10-26","cited_by_count":15,"area":null},{"id":"2206.10461","title":"An Automatic and Efficient BERT Pruning for Edge AI Systems","created":"2022-06-21","cited_by_count":13,"area":null},{"id":"2311.08273","title":"Examining Modularity in Multilingual LMs via Language-Specialized Subnetworks","created":"2023-11-14","cited_by_count":13,"area":null},{"id":"2110.08419","title":"Robustness Challenges in Model Distillation and Pruning for Natural Language Understanding","created":"2021-10-16","cited_by_count":12,"area":null},{"id":"2210.03923","title":"Sparse Teachers Can Be Dense with Knowledge","created":"2022-10-08","cited_by_count":11,"area":null},{"id":"2503.11302","title":"Are formal and functional linguistic mechanisms dissociated in language models?","created":"2025-03-14","cited_by_count":11,"area":null},{"id":"2204.11218","title":"Learning to Win Lottery Tickets in BERT Transfer via Task-agnostic Mask Training","created":"2022-04-24","cited_by_count":10,"area":null},{"id":"2206.15195","title":"The Topological BERT: Transforming Attention into Topology for Natural Language Processing","created":"2022-06-30","cited_by_count":10,"area":null},{"id":"2210.05709","title":"Shapley Head Pruning: Identifying and Removing Interference in Multilingual Transformers","created":"2022-10-11","cited_by_count":9,"area":null},{"id":"2110.03210","title":"Universality of Winning Tickets: A Renormalization Group Perspective","created":"2021-10-07","cited_by_count":8,"area":null},{"id":"2301.12608","title":"Evaluating Neuron Interpretation Methods of NLP Models","created":"2023-01-30","cited_by_count":8,"area":null},{"id":"2306.02320","title":"Exploring the Impact of Model Scaling on Parameter-Efficient Tuning","created":"2023-06-04","cited_by_count":8,"area":null},{"id":"2403.11621","title":"Let's Focus on Neuron: Neuron-Level Supervised Fine-tuning for Large Language Model","created":"2024-03-18","cited_by_count":8,"area":null},{"id":"2407.19126","title":"Greedy Output Approximation: Towards Efficient Structured Pruning for LLMs Without Retraining","created":"2024-07-26","cited_by_count":8,"area":null},{"id":"2103.01819","title":"The Rediscovery Hypothesis: Language Models Need to Meet Linguistics","created":"2021-03-02","cited_by_count":7,"area":null},{"id":"2105.14636","title":"LEAP: Learnable Pruning for Transformer-based Models","created":"2021-05-30","cited_by_count":7,"area":null},{"id":"2205.11833","title":"Diverse Lottery Tickets Boost Ensemble from a Single Pretrained Model","created":"2022-05-24","cited_by_count":7,"area":null},{"id":"2210.04382","title":"Parameter-Efficient Tuning with Special Token Adaptation","created":"2022-10-10","cited_by_count":7,"area":null},{"id":"2210.05211","title":"A Win-win Deal: Towards Sparse and Robust Pre-trained Language Models","created":"2022-10-11","cited_by_count":7,"area":null},{"id":"2211.01484","title":"Data Level Lottery Ticket Hypothesis for Vision Transformers","created":"2022-11-02","cited_by_count":7,"area":null},{"id":"2310.10054","title":"NASH: A Simple Unified Framework of Structured Pruning for Accelerating Encoder-Decoder Language Models","created":"2023-10-16","cited_by_count":7,"area":null},{"id":"2310.13191","title":"Towards Robust Pruning: An Adaptive Knowledge-Retention Pruning Strategy for Language Models","created":"2023-10-19","cited_by_count":7,"area":null},{"id":"2110.04711","title":"SuperShaper: Task-Agnostic Super Pre-training of BERT Models with Variable Hidden Dimensions","created":"2021-10-10","cited_by_count":6,"area":null},{"id":"2101.03255","title":"Spending Your Winning Lottery Better After Drawing It","created":"2021-01-08","cited_by_count":5,"area":null},{"id":"2211.07263","title":"Efficient Adversarial Training with Robust Early-Bird Tickets","created":"2022-11-14","cited_by_count":5,"area":null},{"id":"2302.05045","title":"Exploiting Sparsity in Pruned Neural Networks to Optimize Large Model Training","created":"2023-02-10","cited_by_count":5,"area":null},{"id":"2308.16475","title":"$\\rm SP^3$: Enhancing Structured Pruning via PCA Projection","created":"2023-08-31","cited_by_count":5,"area":null},{"id":"2205.10696","title":"Life after BERT: What do Other Muppets Understand about Language?","created":"2022-05-21","cited_by_count":4,"area":null},{"id":"2207.09638","title":"Doge Tickets: Uncovering Domain-general Language Models by Playing Lottery Tickets","created":"2022-07-20","cited_by_count":4,"area":null},{"id":"2302.12441","title":"MUX-PLMs: Data Multiplexing for High-throughput Language Models","created":"2023-02-24","cited_by_count":4,"area":null},{"id":"2306.01705","title":"The Information Pathways Hypothesis: Transformers are Dynamic Self-Ensembles","created":"2023-06-02","cited_by_count":4,"area":null},{"id":"2403.20284","title":"LayerNorm: A key component in parameter-efficient fine-tuning","created":"2024-03-29","cited_by_count":4,"area":null},{"id":"2603.03415","title":"Farther the Shift, Sparser the Representation: Analyzing OOD Mechanisms in LLMs","created":"2026-03-03","cited_by_count":4,"area":null},{"id":"2109.03939","title":"What's Hidden in a One-layer Randomly Weighted Transformer?","created":"2021-09-08","cited_by_count":3,"area":null},{"id":"2204.02601","title":"Probing Structured Pruning on Multilingual Pre-trained Models: Settings, Algorithms, and Efficiency","created":"2022-04-06","cited_by_count":3,"area":null},{"id":"2301.10835","title":"When Layers Play the Lottery, all Tickets Win at Initialization","created":"2023-01-25","cited_by_count":3,"area":null},{"id":"2305.14706","title":"PruMUX: Augmenting Data Multiplexing with Model Compression","created":"2023-05-24","cited_by_count":3,"area":null},{"id":"2310.13183","title":"Breaking through Deterministic Barriers: Randomized Pruning Mask Generation and Selection","created":"2023-10-19","cited_by_count":3,"area":null},{"id":"2406.02830","title":"Too Big to Fail: Larger Language Models are Disproportionately Resilient to Induction of Dementia-Related Linguistic Anomalies","created":"2024-06-05","cited_by_count":3,"area":null},{"id":"2410.17739","title":"Local Contrastive Editing of Gender Stereotypes","created":"2024-10-23","cited_by_count":3,"area":null},{"id":"2507.03117","title":"BLaST: High Performance Inference and Pretraining using BLock Sparse Transformers","created":"2025-07-03","cited_by_count":3,"area":null},{"id":"2210.13086","title":"Legal-Tech Open Diaries: Lesson learned on how to develop and deploy light-weight models in the era of humongous Language Models","created":"2022-10-24","cited_by_count":2,"area":null},{"id":"2211.00106","title":"Data-Efficient Cross-Lingual Transfer with Language-Specific Subnetworks","created":"2022-10-31","cited_by_count":2,"area":null},{"id":"2302.00378","title":"An Empirical Study on the Transferability of Transformer Modules in Parameter-Efficient Fine-Tuning","created":"2023-02-01","cited_by_count":2,"area":null},{"id":"2510.08513","title":"SliceFine: The Universal Winning-Slice Hypothesis for Pretrained Networks","created":"2025-10-09","cited_by_count":2,"area":null},{"id":"2112.11480","title":"On the Compression of Natural Language Models","created":"2021-12-13","cited_by_count":1,"area":null},{"id":"2203.13802","title":"Playing Lottery Tickets in Style Transfer Models","created":"2022-03-25","cited_by_count":1,"area":null},{"id":"2212.13392","title":"DeepCuts: Single-Shot Interpretability based Pruning for BERT","created":"2022-12-27","cited_by_count":1,"area":null},{"id":"2305.18382","title":"Adaptive Sparsity Level during Training for Efficient Time Series Forecasting with Transformers","created":"2023-05-28","cited_by_count":1,"area":null},{"id":"2310.07911","title":"Pit One Against Many: Leveraging Attention-head Embeddings for Parameter-efficient Multi-head Attention","created":"2023-10-11","cited_by_count":1,"area":null},{"id":"2311.03881","title":"Sparse Contrastive Learning of Sentence Embeddings","created":"2023-11-07","cited_by_count":1,"area":null},{"id":"2404.01306","title":"NeuroPrune: A Neuro-inspired Topological Sparse Training Algorithm for Large Language Models","created":"2024-02-28","cited_by_count":1,"area":null},{"id":"2510.07227","title":"Where to Begin: Efficient Pretraining via Subnetwork Selection and Distillation","created":"2025-10-08","cited_by_count":1,"area":null},{"id":"2510.19778","title":"GaLLoP: Gradient-based Sparse Learning on Low-Magnitude Parameters","created":"2025-10-22","cited_by_count":1,"area":null},{"id":"2511.03983","title":"TwIST: Rigging the Lottery in Transformers with Independent Subnetwork Training","created":"2025-11-06","cited_by_count":1,"area":null},{"id":"2106.07137","title":"Why Can You Lay Off Heads? Investigating How BERT Heads Transfer","created":"2021-06-14","cited_by_count":0,"area":null},{"id":"2111.10951","title":"Can depth-adaptive BERT perform better on binary classification tasks","created":"2021-11-22","cited_by_count":0,"area":null},{"id":"2309.00949","title":"Multilingual Text Representation","created":"2023-09-02","cited_by_count":0,"area":null},{"id":"2312.09541","title":"Picking the Underused Heads: A Network Pruning Perspective of Attention Head Selection for Fusing Dialogue Coreference Information","created":"2023-12-15","cited_by_count":0,"area":null},{"id":"2403.12688","title":"SEVEN: Pruning Transformer Model by Reserving Sentinels","created":"2024-03-19","cited_by_count":0,"area":null}],"next_cursor":"100"}