{"total":17,"rows":[{"id":"2202.08005","title":"Should You Mask 15% in Masked Language Modeling?","created":"2022-02-16","cited_by_count":118,"area":null},{"id":"2407.14962","title":"Recent Advances in Generative AI and Large Language Models: Current Status, Challenges, and Perspectives","created":"2024-07-20","cited_by_count":30,"area":null},{"id":"2205.14912","title":"E2S2: Encoding-Enhanced Sequence-to-Sequence Pretraining for Language Understanding and Generation","created":"2022-05-30","cited_by_count":19,"area":null},{"id":"2203.10415","title":"How does the pre-training objective affect what large language models learn about linguistic properties?","created":"2022-03-20","cited_by_count":16,"area":null},{"id":"2302.09268","title":"Bag of Tricks for Effective Language Model Pretraining and Downstream Adaptation: A Case Study on GLUE","created":"2023-02-18","cited_by_count":11,"area":null},{"id":"2210.07904","title":"HashFormers: Towards Vocabulary-independent Pre-trained Transformers","created":"2022-10-14","cited_by_count":6,"area":null},{"id":"2503.19260","title":"Linguistic Blind Spots of Large Language Models","created":"2025-03-25","cited_by_count":6,"area":null},{"id":"2601.03448","title":"Enhancing Linguistic Competence of Language Models through Pre-training with Language Learning Tasks","created":"2026-01-06","cited_by_count":4,"area":null},{"id":"2210.05471","title":"Instance Regularization for Discriminative Language Model Pre-training","created":"2022-10-11","cited_by_count":2,"area":null},{"id":"2305.10992","title":"How does the task complexity of masked pretraining objectives affect downstream performance?","created":"2023-05-18","cited_by_count":2,"area":null},{"id":"2307.15933","title":"GeneMask: Fast Pretraining of Gene Sequences to Enable Few-Shot Learning","created":"2023-07-29","cited_by_count":2,"area":null},{"id":"2308.08625","title":"BIOptimus: Pre-training an Optimal Biomedical Language Model with Curriculum Learning for Named Entity Recognition","created":"2023-08-16","cited_by_count":2,"area":null},{"id":"2310.17271","title":"Understanding the Role of Input Token Characters in Language Models: How Does Information Loss Affect Performance?","created":"2023-10-26","cited_by_count":2,"area":null},{"id":"2401.07923","title":"Word Boundary Information Isn't Useful for Encoder Language Models","created":"2024-01-15","cited_by_count":2,"area":null},{"id":"2403.10576","title":"Ignore Me But Don't Replace Me: Utilizing Non-Linguistic Elements for Pretraining on the Cybersecurity Domain","created":"2024-03-15","cited_by_count":2,"area":null},{"id":"2212.00460","title":"Language Model Pre-training on True Negatives","created":"2022-12-01","cited_by_count":1,"area":null},{"id":"2301.13455","title":"ZhichunRoad at Amazon KDD Cup 2022: MultiTask Pre-Training for E-Commerce Product Search","created":"2023-01-31","cited_by_count":0,"area":null}],"next_cursor":null}