{"list":"other","total":72,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"1912.01703","title":"PyTorch: An Imperative Style, High-Performance Deep Learning Library","created":"2019-12-03","cited_by_count":20305},{"id":"1804.07461","title":"GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding","created":"2018-04-20","cited_by_count":4899},{"id":"1910.01108","title":"DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter","created":"2019-10-02","cited_by_count":4307},{"id":"1606.05250","title":"SQuAD: 100,000+ Questions for Machine Comprehension of Text","created":"2016-06-16","cited_by_count":4148},{"id":"1606.08415","title":"Gaussian Error Linear Units (GELUs)","created":"2016-06-27","cited_by_count":3502},{"id":"1909.11942","title":"ALBERT: A Lite BERT for Self-supervised Learning of Language Representations","created":"2019-09-26","cited_by_count":2790},{"id":"1704.05426","title":"A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference","created":"2017-04-18","cited_by_count":2682},{"id":"2001.04451","title":"Reformer: The Efficient Transformer","created":"2020-01-13","cited_by_count":1668},{"id":"1806.03822","title":"Know What You Don't Know: Unanswerable Questions for SQuAD","created":"2018-06-11","cited_by_count":1554},{"id":"2006.04768","title":"Linformer: Self-Attention with Linear Complexity","created":"2020-06-08","cited_by_count":1474},{"id":"1909.10351","title":"TinyBERT: Distilling BERT for Natural Language Understanding","created":"2019-09-23","cited_by_count":1167},{"id":"1608.08710","title":"Pruning Filters for Efficient ConvNets","created":"2016-08-31","cited_by_count":1143},{"id":"1708.00055","title":"SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluation","created":"2017-07-31","cited_by_count":1001},{"id":"1805.12471","title":"Neural Network Acceptability Judgments","created":"2018-05-31","cited_by_count":958},{"id":"1905.09418","title":"Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned","created":"2019-05-23","cited_by_count":856},{"id":"1905.10650","title":"Are Sixteen Heads Really Better than One?","created":"2019-05-25","cited_by_count":816},{"id":"1908.09355","title":"Patient Knowledge Distillation for BERT Model Compression","created":"2019-08-25","cited_by_count":508},{"id":"2004.02984","title":"MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices","created":"2020-04-06","cited_by_count":482},{"id":"1902.09574","title":"The State of Sparsity in Deep Neural Networks","created":"2019-02-25","cited_by_count":476},{"id":"1909.11556","title":"Reducing Transformer Depth on Demand with Structured Dropout","created":"2019-09-25","cited_by_count":439},{"id":"2005.07683","title":"Movement Pruning: Adaptive Sparsity by Fine-Tuning","created":"2020-05-15","cited_by_count":403},{"id":"1909.05840","title":"Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT","created":"2019-09-12","cited_by_count":359},{"id":"2004.12993","title":"DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference","created":"2020-04-27","cited_by_count":332},{"id":"2208.11580","title":"Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruning","created":"2022-08-24","cited_by_count":322},{"id":"1910.06188","title":"Q8BERT: Quantized 8Bit BERT","created":"2019-10-14","cited_by_count":300},{"id":"2007.12223","title":"The Lottery Ticket Hypothesis for Pre-trained BERT Networks","created":"2020-07-23","cited_by_count":275},{"id":"2006.04152","title":"BERT Loses Patience: Fast and Robust Inference with Early Exit","created":"2020-06-07","cited_by_count":273},{"id":"2101.01321","title":"I-BERT: Integer-only BERT Quantization","created":"2021-01-05","cited_by_count":255},{"id":"2004.04037","title":"DynaBERT: Dynamic BERT with Adaptive Width and Depth","created":"2020-04-08","cited_by_count":251},{"id":"2012.09852","title":"SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruning","created":"2020-12-17","cited_by_count":234},{"id":"1910.04732","title":"Structured Pruning of Large Language Models","created":"2019-10-10","cited_by_count":203},{"id":"2106.04533","title":"Chasing Sparsity in Vision Transformers: An End-to-End Exploration","created":"2021-06-08","cited_by_count":183},{"id":"1901.11117","title":"The Evolved Transformer","created":"2019-01-30","cited_by_count":180},{"id":"2109.04838","title":"Block Pruning For Faster Transformers","created":"2021-09-10","cited_by_count":177},{"id":"2204.00408","title":"Structured Pruning Learns Compact and Accurate Models","created":"2022-04-01","cited_by_count":170},{"id":"2109.08668","title":"Primer: Searching for Efficient Transformers for Language Modeling","created":"2021-09-17","cited_by_count":164},{"id":"2204.09656","title":"A Fast Post-Training Pruning Framework for Transformers","created":"2022-03-29","cited_by_count":162},{"id":"2005.14187","title":"HAT: Hardware-Aware Transformers for Efficient Natural Language Processing","created":"2020-05-28","cited_by_count":136},{"id":"2004.11886","title":"Lite Transformer with Long-Short Range Attention","created":"2020-04-24","cited_by_count":127},{"id":"2004.03844","title":"On the Effect of Dropping Layers of Pre-trained Transformer Models","created":"2020-04-08","cited_by_count":118},{"id":"2203.07259","title":"The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models","created":"2022-03-14","cited_by_count":115},{"id":"2005.00561","title":"When BERT Plays the Lottery, All Tickets Are Winning","created":"2020-05-01","cited_by_count":114},{"id":"2005.03842","title":"GOBO: Quantizing Attention-Based NLP Models for Low Latency and Energy Efficient Inference","created":"2020-05-08","cited_by_count":106},{"id":"2004.14444","title":"The Effect of Natural Distribution Shift on Question Answering Models","created":"2020-04-29","cited_by_count":104},{"id":"2102.08124","title":"Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Masks","created":"2021-02-16","cited_by_count":103},{"id":"1911.07412","title":"Provable Filter Pruning for Efficient Neural Networks","created":"2019-11-18","cited_by_count":98},{"id":"2002.11985","title":"Compressing Large-Scale Transformer-Based Models: A Case Study on BERT","created":"2020-02-27","cited_by_count":96},{"id":"2006.10518","title":"Improving Post Training Neural Quantization: Layer-wise Calibration and Integer Programming","created":"2020-06-14","cited_by_count":92},{"id":"2101.00063","title":"EarlyBERT: Efficient BERT Training via Early-bird Lottery Tickets","created":"2020-12-31","cited_by_count":79},{"id":"2206.12562","title":"PLATON: Pruning Large Transformer Models with Upper Confidence Bound of Weight Importance","created":"2022-06-25","cited_by_count":79},{"id":"2106.07760","title":"RETRIEVE: Coreset Selection for Efficient and Robust Semi-Supervised Learning","created":"2021-06-14","cited_by_count":74},{"id":"2011.14203","title":"EdgeBERT: Sentence-Level Energy Optimizations for Latency-Aware Multi-Task NLP Inference","created":"2020-11-28","cited_by_count":70},{"id":"2105.14444","title":"NAS-BERT: Task-Agnostic and Adaptive-Size BERT Compression with Neural Architecture Search","created":"2021-05-30","cited_by_count":56},{"id":"2001.04246","title":"AdaBERT: Task-Adaptive BERT Compression with Differentiable Neural Architecture Search","created":"2020-01-13","cited_by_count":51},{"id":"2006.11316","title":"SqueezeBERT: What can computer vision teach NLP about efficient neural networks?","created":"2020-06-19","cited_by_count":50},{"id":"1804.05345","title":"Data-Dependent Coresets for Compressing Neural Networks with Applications to Generalization Bounds","created":"2018-04-15","cited_by_count":43},{"id":"1907.04018","title":"Data-Independent Neural Pruning via Coresets","created":"2019-07-09","cited_by_count":35},{"id":"2008.13006","title":"Accelerating Sparse DNN Models without Hardware-Support via Tile-Wise Sparsity","created":"2020-08-29","cited_by_count":34},{"id":"2107.13686","title":"AutoTinyBERT: Automatic Hyper-parameter Optimization for Efficient Pre-trained Language Models","created":"2021-07-29","cited_by_count":31},{"id":"2006.14284","title":"Fast, Accurate, and Simple Models for Tabular Data via Augmented Distillation","created":"2020-06-25","cited_by_count":27},{"id":"2010.01791","title":"Pruning Redundant Mappings in Transformer Models via Spectral-Normalized Identity Prior","created":"2020-10-05","cited_by_count":23},{"id":"2004.02441","title":"TraDE: Transformers for Density Estimation","created":"2020-04-06","cited_by_count":18},{"id":"2005.06628","title":"schuBERT: Optimizing Elements of BERT","created":"2020-05-09","cited_by_count":17},{"id":"2008.08316","title":"Data-Independent Structured Pruning of Neural Networks via Coresets","created":"2020-08-19","cited_by_count":17},{"id":"2103.11367","title":"ROSITA: Refined BERT cOmpreSsion with InTegrAted techniques","created":"2021-03-21","cited_by_count":14},{"id":"2105.14839","title":"Greedy-layer Pruning: Speeding up Transformer Models for Natural Language Processing","created":"2021-05-31","cited_by_count":11},{"id":"2211.16749","title":"HEAT: Hardware-Efficient Automatic Tensor Decomposition for Transformer Compression","created":"2022-11-30","cited_by_count":9},{"id":"1806.06457","title":"Fast Convex Pruning of Deep Neural Networks","created":"2018-06-17","cited_by_count":8},{"id":"2006.04246","title":"Self-Representation Based Unsupervised Exemplar Selection in a Union of Subspaces","created":"2020-06-07","cited_by_count":8},{"id":"2210.11870","title":"LittleBird: Efficient Faster & Longer Transformer for Question Answering","created":"2022-10-21","cited_by_count":4},{"id":"2205.10403","title":"Tackling Provably Hard Representative Selection via Graph Neural Networks","created":"2022-05-20","cited_by_count":2}],"next_cursor":null}