mlsanityData to 14 Oct 2026

SHARCS: Efficient Transformers through Routing with Dynamic Width Sub-networks

arXiv 2310.12126 ↗First posted 18 Oct 2023cs.LG cs.AI cs.CLcited by 6

Not on the map. Why?

Abstract (arXiv)

We introduce SHARCS for adaptive inference that takes into account the hardness of input samples. SHARCS can train a router on any transformer network, enabling the model to direct different samples to sub-networks with varying widths. Our experiments demonstrate that: (1) SHARCS outperforms or complements existing per-sample adaptive inference methods across various classification tasks in terms of accuracy vs. FLOPs; (2) SHARCS generalizes across different architectures and can be even applied to compressed and efficient transformer encoders to further improve their efficiency; (3) SHARCS can provide a 2 times inference speed up at an insignificant drop in accuracy.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 34 of its 48 references that are in the map. First posted on arXiv 18 Oct 2023.

Builds on

34 of 48 references.

References · 34

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  6. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  7. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  8. 2016Layer Normalizationcited by 5,669
  9. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  10. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  11. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  12. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
22 more
  1. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,167
  2. 2018Neural Network Acceptability Judgmentscited by 958
  3. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 856
  4. 2019Are Sixteen Heads Really Better than One?cited by 816
  5. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 596
  6. 2020MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devicescited by 482
  7. 2020Movement Pruning: Adaptive Sparsity by Fine-Tuningcited by 403
  8. 2019Q-BERT: Hessian Based Ultra Low Precision Quantization of BERTcited by 359
  9. 2020DeeBERT: Dynamic Early Exiting for Accelerating BERT Inferencecited by 332
  10. 2020BERT Loses Patience: Fast and Robust Inference with Early Exitcited by 273
  11. 2021I-BERT: Integer-only BERT Quantizationcited by 255
  12. 2020DynaBERT: Dynamic BERT with Adaptive Width and Depthcited by 251
  13. 2019MetaPruning: Meta Learning for Automatic Neural Network Channel Pruningcited by 227
  14. 2020FastBERT: a Self-distilling BERT with Adaptive Inference Timecited by 226
  15. 2021Block Pruning For Faster Transformerscited by 177
  16. 2022Structured Pruning Learns Compact and Accurate Modelscited by 170
  17. 2020The Right Tool for the Job: Matching Model and Instance Complexitiescited by 126
  18. 2017Bilateral Multi-Perspective Matching for Natural Language Sentencescited by 121
  19. 2023MatFormer: Nested Transformer for Elastic Inferencecited by 85
  20. 2022A Simple Hash-Based Early Exiting Approach For Language Understanding and Generationcited by 32
  21. 2022Transkimmer: Transformer Learns to Layer-wise Skimcited by 24
  22. 2018Shallow-Deep Networks: Understanding and Mitigating Network Overthinkingcited by 7

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 2; 2025: 3.

6 papers.

  1. 2023OrchestraLLM: Efficient Orchestration of Language Models for Dialogue State Trackingcited by 19
  2. 2024HydraViT: Stacking Heads for a Scalable ViTcited by 11
  3. 2024Duo-LLM: A Framework for Studying Adaptive Computation in Large Language Modelscited by 7
  4. 2025From Dense to Dynamic: Token-Difficulty Driven MoEfication of Pre-Trained LLMscited by 5
  5. 2025Efficiency Robustness of Dynamic Deep Learning Systemscited by 1
  6. 2025AdaVid: Adaptive Video-Language Pretrainingcited by 0