{"total":132,"rows":[{"id":"2104.11227","title":"Multiscale Vision Transformers","created":"2021-04-22","cited_by_count":897,"area":null},{"id":"2106.04803","title":"CoAtNet: Marrying Convolution and Attention for All Data Sizes","created":"2021-06-09","cited_by_count":569,"area":null},{"id":"2106.04554","title":"A Survey of Transformers","created":"2021-06-08","cited_by_count":401,"area":null},{"id":"2103.16302","title":"Rethinking Spatial Dimensions of Vision Transformers","created":"2021-03-30","cited_by_count":361,"area":null},{"id":"2206.13947","title":"Long Range Language Modeling via Gated State Spaces","created":"2022-06-27","cited_by_count":265,"area":null},{"id":"2103.06874","title":"CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representation","created":"2021-03-11","cited_by_count":232,"area":null},{"id":"2312.03863","title":"Efficient Large Language Models: A Survey","created":"2023-12-06","cited_by_count":186,"area":null},{"id":"2104.08253","title":"Condenser: a Pre-training Architecture for Dense Retrieval","created":"2021-04-16","cited_by_count":162,"area":null},{"id":"2106.12672","title":"Charformer: Fast Character Transformers via Gradient-based Subword Tokenization","created":"2021-06-23","cited_by_count":160,"area":null},{"id":"2203.07852","title":"Block-Recurrent Transformers","created":"2022-03-11","cited_by_count":110,"area":null},{"id":"2207.10551","title":"Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling?","created":"2022-07-21","cited_by_count":103,"area":null},{"id":"2110.13711","title":"Hierarchical Transformers Are More Efficient Language Models","created":"2021-10-26","cited_by_count":102,"area":null},{"id":"2103.13076","title":"Finetuning Pretrained Transformers into RNNs","created":"2021-03-24","cited_by_count":96,"area":null},{"id":"2103.14304","title":"Exploiting Temporal Contexts with Strided Transformer for 3D Human Pose Estimation","created":"2021-03-26","cited_by_count":94,"area":null},{"id":"2102.11972","title":"Do Transformer Modifications Transfer Across Implementations and Applications?","created":"2021-02-23","cited_by_count":87,"area":null},{"id":"2103.10619","title":"Scalable Vision Transformers with Hierarchical Pooling","created":"2021-03-19","cited_by_count":84,"area":null},{"id":"2210.14199","title":"Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models","created":"2022-10-25","cited_by_count":84,"area":null},{"id":"2203.16329","title":"Parameter-efficient Model Adaptation for Vision Transformers","created":"2022-03-29","cited_by_count":81,"area":null},{"id":"2212.14034","title":"Cramming: Training a Language Model on a Single GPU in One Day","created":"2022-12-28","cited_by_count":81,"area":null},{"id":"2211.09761","title":"Efficient Transformers with Dynamic Token Pooling","created":"2022-11-17","cited_by_count":76,"area":null},{"id":"2103.14636","title":"A Practical Survey on Faster and Lighter Transformers","created":"2021-03-26","cited_by_count":73,"area":null},{"id":"2507.07955","title":"Dynamic Chunking for End-to-End Hierarchical Sequence Modeling","created":"2025-07-10","cited_by_count":72,"area":null},{"id":"2305.15805","title":"Dynamic Context Pruning for Efficient and Interpretable Autoregressive Transformers","created":"2023-05-25","cited_by_count":67,"area":null},{"id":"2202.07765","title":"General-purpose, long-context autoregressive modeling with Perceiver AR","created":"2022-02-15","cited_by_count":60,"area":null},{"id":"2204.13767","title":"Triformer: Triangular, Variable-Specific Attentions for Long Sequence Multivariate Time Series Forecasting--Full Version","created":"2022-04-28","cited_by_count":59,"area":null},{"id":"2406.16747","title":"Sparser is Faster and Less is More: Efficient Sparse Attention for Long-Range Transformers","created":"2024-06-24","cited_by_count":52,"area":null},{"id":"2109.06067","title":"Packed Levitated Marker for Entity and Relation Extraction","created":"2021-09-13","cited_by_count":49,"area":null},{"id":"2203.06667","title":"Towards Visual-Prompt Temporal Answering Grounding in Medical Instructional Video","created":"2022-03-13","cited_by_count":43,"area":null},{"id":"2412.09604","title":"SynerGen-VL: Towards Synergistic Image Understanding and Generation with Vision Experts and Token Folding","created":"2024-12-12","cited_by_count":42,"area":null},{"id":"2108.01250","title":"Your fairness may vary: Pretrained language model fairness in toxic text classification","created":"2021-08-03","cited_by_count":39,"area":null},{"id":"2107.07999","title":"From block-Toeplitz matrices to differential equations on graphs: towards a general theory for scalable masked Transformers","created":"2021-07-16","cited_by_count":38,"area":null},{"id":"2104.08698","title":"A Simple and Effective Positional Encoding for Transformers","created":"2021-04-18","cited_by_count":37,"area":null},{"id":"2104.01655","title":"Efficient Transformers in Reinforcement Learning using Actor-Learner Distillation","created":"2021-04-04","cited_by_count":35,"area":null},{"id":"2404.14408","title":"SpaceByte: Towards Deleting Tokenization from Large Language Modeling","created":"2024-04-22","cited_by_count":34,"area":null},{"id":"2311.05908","title":"FlashFFTConv: Efficient Convolutions for Long Sequences with Tensor Cores","created":"2023-11-10","cited_by_count":33,"area":null},{"id":"2204.04667","title":"Linear Complexity Randomized Self-attention Mechanism","created":"2022-04-10","cited_by_count":31,"area":null},{"id":"2104.06952","title":"The Surprising Performance of Simple Baselines for Misinformation Detection","created":"2021-04-14","cited_by_count":28,"area":null},{"id":"2105.04297","title":"How could Neural Networks understand Programs?","created":"2021-05-10","cited_by_count":26,"area":null},{"id":"2112.11294","title":"Extending CLIP for Category-to-image Retrieval in E-commerce","created":"2021-12-21","cited_by_count":24,"area":null},{"id":"2307.03407","title":"Distilling Self-Supervised Vision Transformers for Weakly-Supervised Few-Shot Classification & Segmentation","created":"2023-07-07","cited_by_count":24,"area":null},{"id":"2302.04542","title":"Efficient Attention via Control Variates","created":"2023-02-09","cited_by_count":23,"area":null},{"id":"2107.08031","title":"Is attention to bounding boxes all you need for pedestrian action prediction?","created":"2021-07-16","cited_by_count":21,"area":null},{"id":"2305.15099","title":"Fourier Transformer: Fast Long Range Modeling by Removing Sequence Redundancy with FFT Operator","created":"2023-05-24","cited_by_count":20,"area":null},{"id":"2202.02842","title":"Evaluating natural language processing models with generalization metrics that do not need access to any training or testing data","created":"2022-02-06","cited_by_count":19,"area":null},{"id":"2112.08560","title":"Block-Skim: Efficient Question Answering for Transformer","created":"2021-12-16","cited_by_count":18,"area":null},{"id":"2308.06294","title":"Enhancing Phenotype Recognition in Clinical Notes Using Large Language Models: PhenoBCBERT and PhenoGPT","created":"2023-08-11","cited_by_count":18,"area":null},{"id":"2303.10757","title":"Multiscale Audio Spectrogram Transformer for Efficient Audio Classification","created":"2023-03-19","cited_by_count":16,"area":null},{"id":"2305.12689","title":"FIT: Far-reaching Interleaved Transformers","created":"2023-05-22","cited_by_count":16,"area":null},{"id":"2202.10890","title":"HiP: Hierarchical Perceiver","created":"2022-02-22","cited_by_count":15,"area":null},{"id":"2401.15347","title":"A Comprehensive Survey of Compression Algorithms for Language Models","created":"2024-01-27","cited_by_count":15,"area":null},{"id":"2203.14380","title":"Pyramid-BERT: Reducing Complexity via Successive Core-set based Token Selection","created":"2022-03-27","cited_by_count":13,"area":null},{"id":"2301.13310","title":"Alternating Updates for Efficient Transformers","created":"2023-01-30","cited_by_count":13,"area":null},{"id":"2106.09204","title":"An Empirical Study on Hyperparameter Optimization for Fine-Tuning Pre-trained Language Models","created":"2021-06-17","cited_by_count":12,"area":null},{"id":"2204.06164","title":"A Unified Cascaded Encoder ASR Model for Dynamic Model Sizes","created":"2022-04-13","cited_by_count":11,"area":null},{"id":"2302.06117","title":"The Framework Tax: Disparities Between Inference Efficiency in NLP Research and Deployment","created":"2023-02-13","cited_by_count":11,"area":null},{"id":"2210.07661","title":"CAB: Comprehensive Attention Benchmarking on Long Sequence Modeling","created":"2022-10-14","cited_by_count":10,"area":null},{"id":"2305.00581","title":"Multimodal Graph Transformer for Multimodal Question Answering","created":"2023-04-30","cited_by_count":10,"area":null},{"id":"2203.02898","title":"Divide and Conquer: Text Semantic Matching with Disentangled Keywords and Intents","created":"2022-03-06","cited_by_count":9,"area":null},{"id":"2203.10282","title":"Clickbait Spoiling via Question Answering and Passage Retrieval","created":"2022-03-19","cited_by_count":9,"area":null},{"id":"2204.11458","title":"ED2LM: Encoder-Decoder to Language Model for Faster Document Re-ranking Inference","created":"2022-04-25","cited_by_count":9,"area":null},{"id":"2309.08208","title":"HM-Conformer: A Conformer-based audio deepfake detection system with hierarchical pooling and multi-level classification token aggregation methods","created":"2023-09-15","cited_by_count":8,"area":null},{"id":"2311.08620","title":"Toucan: Token-Aware Character Level Language Modeling","created":"2023-11-15","cited_by_count":8,"area":null},{"id":"2506.00450","title":"DV365: Extremely Long User History Modeling at Instagram","created":"2025-05-31","cited_by_count":8,"area":null},{"id":"2403.03344","title":"Learn to Code Sustainably: An Empirical Study on LLM-based Green Code Generation","created":"2024-03-05","cited_by_count":7,"area":null},{"id":"2408.11871","title":"MegaFake: A Theory-Driven Dataset of Fake News Generated by Large Language Models","created":"2024-08-19","cited_by_count":7,"area":null},{"id":"2208.03985","title":"Generating Coherent Narratives by Learning Dynamic and Discrete Entity States with a Contrastive Framework","created":"2022-08-08","cited_by_count":6,"area":null},{"id":"2208.13363","title":"Survey: Exploiting Data Redundancy for Optimization of Deep Learning","created":"2022-08-29","cited_by_count":6,"area":null},{"id":"2305.04241","title":"Vcc: Scaling Transformers to 128K Tokens or More by Prioritizing Important Tokens","created":"2023-05-07","cited_by_count":6,"area":null},{"id":"2305.11392","title":"Fast-StrucTexT: An Efficient Hourglass Transformer with Modality-guided Dynamic Token Merge for Document Understanding","created":"2023-05-19","cited_by_count":6,"area":null},{"id":"2307.14132","title":"CIF-T: A Novel CIF-based Transducer Architecture for Automatic Speech Recognition","created":"2023-07-26","cited_by_count":6,"area":null},{"id":"2402.17184","title":"Extreme Encoder Output Frame Rate Reduction: Improving Computational Latencies of Large End-to-End Models","created":"2024-02-27","cited_by_count":6,"area":null},{"id":"2106.06731","title":"Incorporating External POS Tagger for Punctuation Restoration","created":"2021-06-12","cited_by_count":5,"area":null},{"id":"2211.14591","title":"A Survey of Text Representation Methods and Their Genealogy","created":"2022-11-26","cited_by_count":5,"area":null},{"id":"2306.01824","title":"Enhancing the Protein Tertiary Structure Prediction by Multiple Sequence Alignment Generation","created":"2023-06-02","cited_by_count":5,"area":null},{"id":"2307.07166","title":"Switching Head-Tail Funnel UNITER for Dual Referring Expression Comprehension with Fetch-and-Carry Tasks","created":"2023-07-14","cited_by_count":5,"area":null},{"id":"2311.04449","title":"Recursion in Recursion: Two-Level Nested Recursion for Length Generalization with Scalability","created":"2023-11-08","cited_by_count":5,"area":null},{"id":"2404.10180","title":"Deferred NAM: Low-latency Top-K Context Injection via Deferred Context Encoding for Non-Streaming ASR","created":"2024-04-15","cited_by_count":5,"area":null},{"id":"2111.09645","title":"Dynamic-TinyBERT: Boost TinyBERT's Inference Efficiency by Dynamic Sequence Length","created":"2021-11-18","cited_by_count":4,"area":null},{"id":"2205.01293","title":"A Survey of Deep Learning Models for Structural Code Understanding","created":"2022-05-03","cited_by_count":4,"area":null},{"id":"2312.02429","title":"PEFA: Parameter-Free Adapters for Large-scale Embedding-based Retrieval Models","created":"2023-12-05","cited_by_count":4,"area":null},{"id":"2107.13054","title":"Exceeding the Limits of Visual-Linguistic Multi-Task Learning","created":"2021-07-27","cited_by_count":3,"area":null},{"id":"2301.04761","title":"NarrowBERT: Accelerating Masked Language Model Pretraining and Inference","created":"2023-01-11","cited_by_count":3,"area":null},{"id":"2306.01070","title":"Hierarchical Attention Encoder Decoder","created":"2023-06-01","cited_by_count":3,"area":null},{"id":"2404.13390","title":"Explanation based Bias Decoupling Regularization for Natural Language Inference","created":"2024-04-20","cited_by_count":3,"area":null},{"id":"2406.02921","title":"Text Injection for Neural Contextual Biasing","created":"2024-06-05","cited_by_count":3,"area":null},{"id":"2504.02877","title":"Revisiting Funnel Transformers for Modern LLM Architectures with Comprehensive Ablations in Training and Inference Configurations","created":"2025-04-02","cited_by_count":3,"area":null},{"id":"2203.12487","title":"A Context-Aware Feature Fusion Framework for Punctuation Restoration","created":"2022-03-23","cited_by_count":2,"area":null},{"id":"2210.08475","title":"RedApt: An Adaptor for wav2vec 2 Encoding \\\\ Faster and Smaller Speech Translation without Quality Compromise","created":"2022-10-16","cited_by_count":2,"area":null},{"id":"2305.17356","title":"Bridging the Granularity Gap for Acoustic Modeling","created":"2023-05-27","cited_by_count":2,"area":null},{"id":"2308.02727","title":"How Good Are SOTA Fake News Detectors","created":"2023-08-04","cited_by_count":2,"area":null},{"id":"2310.08085","title":"Low-Resource Clickbait Spoiling for Indonesian via Question Answering","created":"2023-10-12","cited_by_count":2,"area":null},{"id":"2312.02125","title":"TPPoet: Transformer-Based Persian Poem Generation using Minimal Data and Advanced Decoding Techniques","created":"2023-12-04","cited_by_count":2,"area":null},{"id":"2402.01416","title":"Sequence Shortening for Context-Aware Machine Translation","created":"2024-02-02","cited_by_count":2,"area":null},{"id":"2408.08567","title":"S$^3$Attention: Improving Long Sequence Attention with Smoothed Skeleton Sketching","created":"2024-08-16","cited_by_count":2,"area":null},{"id":"2410.23510","title":"Tiny Transformers Excel at Sentence Compression","created":"2024-10-30","cited_by_count":2,"area":null},{"id":"2506.02939","title":"QKV Projections Require a Fraction of Their Memory","created":"2025-06-03","cited_by_count":2,"area":null},{"id":"2606.23670","title":"Tapered Language Models","created":"2026-06-22","cited_by_count":2,"area":null},{"id":"2201.03327","title":"Latency Adjustable Transformer Encoder for Language Understanding","created":"2022-01-10","cited_by_count":1,"area":null},{"id":"2201.04406","title":"GateFormer: Speeding Up News Feed Recommendation with Input Gated Transformers","created":"2022-01-12","cited_by_count":1,"area":null},{"id":"2211.07384","title":"Language models are good pathologists: using attention-based sequence reduction and text-pretrained transformers for efficient WSI classification","created":"2022-11-14","cited_by_count":1,"area":null}],"next_cursor":"100"}