{"total":54,"rows":[{"id":"2202.08005","title":"Should You Mask 15% in Masked Language Modeling?","created":"2022-02-16","cited_by_count":118,"area":null},{"id":"2106.09226","title":"Why Do Pretrained Language Models Help in Downstream Tasks? An Analysis of Head and Prompt Tuning","created":"2021-06-17","cited_by_count":82,"area":null},{"id":"2110.08499","title":"PRIMERA: Pyramid-based Masked Sentence Pre-training for Multi-document Summarization","created":"2021-10-16","cited_by_count":62,"area":null},{"id":"2406.09406","title":"4M-21: An Any-to-Any Vision Model for Tens of Tasks and Modalities","created":"2024-06-13","cited_by_count":52,"area":null},{"id":"2310.14753","title":"Rethinking Tokenizer and Decoder in Masked Graph Modeling for Molecules","created":"2023-10-23","cited_by_count":49,"area":null},{"id":"2301.06568","title":"Ankh: Optimized Protein Language Model Unlocks General-Purpose Modelling","created":"2023-01-16","cited_by_count":45,"area":null},{"id":"2106.02902","title":"BERTnesia: Investigating the capture and forgetting of knowledge in BERT","created":"2021-06-05","cited_by_count":40,"area":null},{"id":"2109.10246","title":"Does Vision-and-Language Pretraining Improve Lexical Grounding?","created":"2021-09-21","cited_by_count":25,"area":null},{"id":"2211.10869","title":"UniMASK: Unified Inference in Sequential Decision Problems","created":"2022-11-20","cited_by_count":24,"area":null},{"id":"2202.06387","title":"Scaling Laws Under the Microscope: Predicting Transformer Performance from Small Scale Experiments","created":"2022-02-13","cited_by_count":23,"area":null},{"id":"2109.02040","title":"Data Efficient Masked Language Modeling for Vision and Language","created":"2021-09-05","cited_by_count":19,"area":"mllm"},{"id":"2105.03928","title":"Which transformer architecture fits my data? A vocabulary bottleneck in self-attention","created":"2021-05-09","cited_by_count":16,"area":null},{"id":"2208.10806","title":"Learning Better Masking for Better Language Model Pre-training","created":"2022-08-23","cited_by_count":15,"area":null},{"id":"2210.11771","title":"InforMask: Unsupervised Informative Masking for Language Model Pretraining","created":"2022-10-21","cited_by_count":15,"area":null},{"id":"2102.01909","title":"HeBERT & HebEMO: a Hebrew BERT Model and a Tool for Polarity Analysis and Emotion Recognition","created":"2021-02-03","cited_by_count":14,"area":null},{"id":"2303.12860","title":"Salient Span Masking for Temporal Understanding","created":"2023-03-22","cited_by_count":14,"area":null},{"id":"2104.08840","title":"On the Influence of Masking Policies in Intermediate Pre-training","created":"2021-04-18","cited_by_count":12,"area":null},{"id":"2212.07617","title":"Efficient Pre-training of Masked Language Model via Concept-based Curriculum Masking","created":"2022-12-15","cited_by_count":12,"area":null},{"id":"2305.12816","title":"Farewell to Aimless Large-scale Pretraining: Influential Subset Selection for Language Model","created":"2023-05-22","cited_by_count":12,"area":null},{"id":"2305.15096","title":"Dynamic Masking Rate Schedules for MLM Pretraining","created":"2023-05-24","cited_by_count":12,"area":null},{"id":"2205.04515","title":"Unsupervised Slot Schema Induction for Task-oriented Dialog","created":"2022-05-09","cited_by_count":11,"area":null},{"id":"2309.10738","title":"MelodyGLM: Multi-task Pre-training for Symbolic Melody Generation","created":"2023-09-19","cited_by_count":10,"area":null},{"id":"2305.15275","title":"Self-Evolution Learning for Discriminative Language Model Pretraining","created":"2023-05-24","cited_by_count":8,"area":null},{"id":"2204.13326","title":"Towards Flexible Inference in Sequential Decision Problems via Bidirectional Transformers","created":"2022-04-28","cited_by_count":6,"area":null},{"id":"2305.05461","title":"What is the best recipe for character-level encoder-only modelling?","created":"2023-05-09","cited_by_count":6,"area":null},{"id":"2412.17534","title":"CiteBART: Learning to Generate Citations for Local Citation Recommendation","created":"2024-12-23","cited_by_count":6,"area":null},{"id":"2401.17514","title":"How Useful is Continued Pre-Training for Generative Unsupervised Domain Adaptation?","created":"2024-01-31","cited_by_count":5,"area":null},{"id":"2507.12838","title":"Are Knowledge and Reference in Multilingual Language Models Cross-Lingually Consistent?","created":"2025-07-17","cited_by_count":5,"area":null},{"id":"2210.11165","title":"Pre-training Language Models with Deterministic Factual Knowledge","created":"2022-10-20","cited_by_count":4,"area":null},{"id":"2506.02584","title":"Prosodic Structure Beyond Lexical Content: A Study of Self-Supervised Learning","created":"2025-06-03","cited_by_count":4,"area":null},{"id":"2202.12719","title":"Ask2Mask: Guided Data Selection for Masked Speech Modeling","created":"2022-02-24","cited_by_count":3,"area":null},{"id":"2304.10973","title":"LEIA: Linguistic Embeddings for the Identification of Affect","created":"2023-04-21","cited_by_count":3,"area":null},{"id":"2305.13197","title":"Challenging Decoder helps in Masked Auto-Encoder Pre-training for Dense Passage Retrieval","created":"2023-05-22","cited_by_count":3,"area":null},{"id":"2509.21650","title":"Understanding and Enhancing Mask-Based Pretraining towards Universal Representations","created":"2025-09-25","cited_by_count":3,"area":null},{"id":"2205.11456","title":"Multilingual Extraction and Categorization of Lexical Collocations with Graph-aware Transformers","created":"2022-05-23","cited_by_count":2,"area":null},{"id":"2210.05471","title":"Instance Regularization for Discriminative Language Model Pre-training","created":"2022-10-11","cited_by_count":2,"area":null},{"id":"2307.15933","title":"GeneMask: Fast Pretraining of Gene Sequences to Enable Few-Shot Learning","created":"2023-07-29","cited_by_count":2,"area":null},{"id":"2402.12036","title":"Language Model Adaptation to Specialized Domains through Selective Masking based on Genre and Topical Characteristics","created":"2024-02-19","cited_by_count":2,"area":null},{"id":"2408.07180","title":"Unlocking Efficiency: Adaptive Masking for Gene Transformer Models","created":"2024-08-13","cited_by_count":2,"area":null},{"id":"2509.05056","title":"Masked Diffusion Language Models with Frequency-Informed Training","created":"2025-09-05","cited_by_count":2,"area":"diffusion"},{"id":"2212.00460","title":"Language Model Pre-training on True Negatives","created":"2022-12-01","cited_by_count":1,"area":null},{"id":"2212.05195","title":"Uniform Masking Prevails in Vision-Language Pretraining","created":"2022-12-10","cited_by_count":1,"area":"mllm"},{"id":"2408.01426","title":"MolTRES: Improving Chemical Language Representation Learning for Molecular Property Prediction","created":"2024-07-09","cited_by_count":1,"area":null},{"id":"2501.05700","title":"Linguistic Entity Masking to Improve Cross-Lingual Representation of Multilingual Language Models for Low-Resource Languages","created":"2025-01-10","cited_by_count":1,"area":null},{"id":"2505.04643","title":"Prediction-powered estimators for finite population statistics in highly imbalanced textual data: Public hate crime estimation","created":"2025-05-05","cited_by_count":1,"area":null},{"id":"2207.03858","title":"DSTEA: Improving Dialogue State Tracking via Entity Adaptive Pre-training","created":"2022-07-08","cited_by_count":0,"area":null},{"id":"2303.15619","title":"Typhoon: Towards an Effective Task-Specific Masking Strategy for Pre-trained Language Models","created":"2023-03-27","cited_by_count":0,"area":null},{"id":"2306.07185","title":"The Effect of Masking Strategies on Knowledge Retention by Language Models","created":"2023-06-12","cited_by_count":0,"area":null},{"id":"2403.18025","title":"Improving Pre-trained Language Model Sensitivity via Mask Specific losses: A case study on Biomedical NER","created":"2024-03-26","cited_by_count":0,"area":null},{"id":"2405.07788","title":"DEPTH: Discourse Education through Pre-Training Hierarchically","created":"2024-05-13","cited_by_count":0,"area":null},{"id":"2410.24201","title":"LingGen: Scalable Multi-Attribute Linguistic Control via Power-Law Masking","created":"2024-10-31","cited_by_count":0,"area":null},{"id":"2411.10172","title":"Increasing the Accessibility of Causal Domain Knowledge via Causal Information Extraction Methods: A Case Study in the Semiconductor Manufacturing Industry","created":"2024-11-15","cited_by_count":0,"area":null},{"id":"2512.05216","title":"Coefficient of Variation Masking: A Volatility-Aware Strategy for EHR Foundation Models","created":"2025-12-04","cited_by_count":0,"area":null},{"id":"2606.22079","title":"Where Does the Signal Live? A Web Data Recipe for Medical Encoder Pretraining","created":"2026-06-20","cited_by_count":0,"area":null}],"next_cursor":null}