{"total":184,"rows":[{"id":"2303.17564","title":"BloombergGPT: A Large Language Model for Finance","created":"2023-03-30","cited_by_count":939,"area":null},{"id":"2107.14795","title":"Perceiver IO: A General Architecture for Structured Inputs & Outputs","created":"2021-07-30","cited_by_count":629,"area":null},{"id":"2103.06874","title":"CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representation","created":"2021-03-11","cited_by_count":232,"area":null},{"id":"2305.13707","title":"Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Models","created":"2023-05-23","cited_by_count":192,"area":null},{"id":"2106.12672","title":"Charformer: Fast Character Transformers via Gradient-based Subword Tokenization","created":"2021-06-23","cited_by_count":160,"area":null},{"id":"2112.10508","title":"Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP","created":"2021-12-20","cited_by_count":132,"area":null},{"id":"2402.14903","title":"Tokenization counts: the impact of tokenization on arithmetic in frontier LLMs","created":"2024-02-22","cited_by_count":115,"area":null},{"id":"2310.08754","title":"Tokenizer Choice For LLM Training: Negligible or Crucial?","created":"2023-10-12","cited_by_count":108,"area":null},{"id":"2402.18376","title":"Tokenization Is More Than Compression","created":"2024-02-28","cited_by_count":89,"area":null},{"id":"2404.00925","title":"LLMs are Good Sign Language Translators","created":"2024-04-01","cited_by_count":77,"area":null},{"id":"2211.09761","title":"Efficient Transformers with Dynamic Token Pooling","created":"2022-11-17","cited_by_count":76,"area":null},{"id":"2101.00403","title":"Superbizarre Is Not Superb: Derivational Morphology Improves BERT's Interpretation of Complex Words","created":"2021-01-02","cited_by_count":63,"area":null},{"id":"2204.08832","title":"Impact of Tokenization on Language Models: An Analysis for Turkish","created":"2022-04-19","cited_by_count":62,"area":null},{"id":"2411.14198","title":"Why do language models perform worse for morphologically complex languages?","created":"2024-11-21","cited_by_count":62,"area":null},{"id":"2403.06265","title":"Unpacking Tokenization: Evaluating Text Compression and its Correlation with Model Performance","created":"2024-03-10","cited_by_count":61,"area":null},{"id":"2207.06991","title":"Language Modelling with Pixels","created":"2022-07-14","cited_by_count":55,"area":null},{"id":"2406.04303","title":"Vision-LSTM: xLSTM as Generic Vision Backbone","created":"2024-06-06","cited_by_count":51,"area":null},{"id":"2510.06265","title":"Large Language Models Hallucination: A Comprehensive Survey","created":"2025-10-05","cited_by_count":49,"area":null},{"id":"2410.05864","title":"From Tokens to Words: On the Inner Lexicon of LLMs","created":"2024-10-08","cited_by_count":48,"area":null},{"id":"2405.05417","title":"Fishing for Magikarp: Automatically Detecting Under-trained Tokens in Large Language Models","created":"2024-05-08","cited_by_count":46,"area":null},{"id":"2103.08490","title":"Multi-view Subword Regularization","created":"2021-03-15","cited_by_count":40,"area":null},{"id":"2106.13474","title":"Adapt-and-Distill: Developing Small, Fast and Effective Pretrained Language Models for Domains","created":"2021-06-25","cited_by_count":36,"area":null},{"id":"2306.16837","title":"A Formal Perspective on Byte-Pair Encoding","created":"2023-06-29","cited_by_count":36,"area":null},{"id":"2403.01289","title":"Greed is All You Need: An Evaluation of Tokenizer Inference Methods","created":"2024-03-02","cited_by_count":34,"area":null},{"id":"2212.10502","title":"A Measure-Theoretic Characterization of Tight Language Models","created":"2022-12-20","cited_by_count":33,"area":null},{"id":"2504.00178","title":"Boundless Byte Pair Encoding: Breaking the Pre-tokenization Barrier","created":"2025-03-31","cited_by_count":32,"area":null},{"id":"2409.04599","title":"BPE Gets Picky: Efficient Vocabulary Refinement During Tokenizer Training","created":"2024-09-06","cited_by_count":31,"area":null},{"id":"2109.02550","title":"You should evaluate your language model on marginal likelihood over tokenisations","created":"2021-09-06","cited_by_count":30,"area":null},{"id":"2310.13348","title":"Analyzing Cognitive Plausibility of Subword Tokenization","created":"2023-10-20","cited_by_count":30,"area":null},{"id":"2206.09755","title":"Square One Bias in NLP: Towards a Multi-Dimensional Exploration of the Research Manifold","created":"2022-06-20","cited_by_count":29,"area":null},{"id":"2406.17681","title":"VarBench: Robust Language Model Benchmarking Through Dynamic Variable Perturbation","created":"2024-06-25","cited_by_count":29,"area":null},{"id":"2305.12520","title":"SLaDe: A Portable Small Language Model Decompiler for Optimized Assembly","created":"2023-05-21","cited_by_count":28,"area":null},{"id":"2303.05759","title":"An Overview on Language Models: Recent Developments and Outlook","created":"2023-03-10","cited_by_count":22,"area":null},{"id":"2406.11477","title":"How Can We Effectively Expand the Vocabulary of LLMs with 0.01GB of Target Language Text?","created":"2024-06-17","cited_by_count":22,"area":null},{"id":"2407.11606","title":"The Foundations of Tokenization: Statistical and Computational Concerns","created":"2024-07-16","cited_by_count":22,"area":null},{"id":"2507.06378","title":"Evaluating Morphological Alignment of Tokenizers in 70 Languages","created":"2025-07-08","cited_by_count":22,"area":null},{"id":"2204.14268","title":"How Robust is Neural Machine Translation to Language Imbalance in Multilingual Tokenizer Training?","created":"2022-04-29","cited_by_count":20,"area":null},{"id":"2306.09572","title":"How do different tokenizers perform on downstream tasks in scriptio continua languages?: A case study in Japanese","created":"2023-06-16","cited_by_count":20,"area":null},{"id":"2508.04796","title":"Parity-Aware Byte-Pair Encoding: Improving Cross-lingual Fairness in Tokenization","created":"2025-08-06","cited_by_count":20,"area":null},{"id":"2204.04058","title":"Improving Tokenisation by Alternative Treatment of Spaces","created":"2022-04-08","cited_by_count":19,"area":null},{"id":"2204.11909","title":"How can NLP Help Revitalize Endangered Languages? A Case Study and Roadmap for the Cherokee Language","created":"2022-04-25","cited_by_count":19,"area":null},{"id":"2301.11975","title":"Byte Pair Encoding for Symbolic Music","created":"2023-01-27","cited_by_count":19,"area":null},{"id":"2410.12656","title":"Evaluating Morphological Compositional Generalization in Large Language Models","created":"2024-10-16","cited_by_count":19,"area":null},{"id":"2402.10712","title":"An Empirical Study on Cross-lingual Vocabulary Adaptation for Efficient Language Model Inference","created":"2024-02-16","cited_by_count":18,"area":null},{"id":"2404.03626","title":"Training LLMs over Neurally Compressed Text","created":"2024-04-04","cited_by_count":17,"area":null},{"id":"2212.09897","title":"Inducing Character-level Structure in Subword-based Language Models with Type-level Interchange Intervention Training","created":"2022-12-19","cited_by_count":16,"area":null},{"id":"2312.11779","title":"Tokenization Matters: Navigating Data-Scarce Tokenization for Gender Inclusive Language Technologies","created":"2023-12-19","cited_by_count":16,"area":null},{"id":"2411.18553","title":"Retrofitting Large Language Models with Dynamic Tokenization","created":"2024-11-27","cited_by_count":16,"area":null},{"id":"2105.03484","title":"Empirical Evaluation of Pre-trained Transformers for Human-Level NLP: The Role of Sample Size and Dimensionality","created":"2021-05-07","cited_by_count":15,"area":null},{"id":"2106.07540","title":"Evaluating Various Tokenizers for Arabic Text Classification","created":"2021-06-14","cited_by_count":14,"area":null},{"id":"2112.14569","title":"Fine-Tuning Transformers: Vocabulary Transfer","created":"2021-12-29","cited_by_count":14,"area":null},{"id":"2511.01380","title":"Confounding Factors in Relating Model Performance to Morphology","created":"2025-11-03","cited_by_count":14,"area":null},{"id":"2505.21627","title":"Is Your LLM Overcharging You? Tokenization, Transparency, and Incentives","created":"2025-05-27","cited_by_count":13,"area":null},{"id":"2204.10815","title":"A Vocabulary-Free Multilingual Neural Tokenizer for End-to-End Task Learning","created":"2022-04-22","cited_by_count":12,"area":null},{"id":"2305.05480","title":"Effects of sub-word segmentation on performance of transformer language models","created":"2023-05-09","cited_by_count":12,"area":null},{"id":"2412.13716","title":"Model Decides How to Tokenize: Adaptive DNA Sequence Tokenization with MxDNA","created":"2024-12-18","cited_by_count":12,"area":null},{"id":"2502.11596","title":"LLM Embeddings for Deep Learning on Tabular Data","created":"2025-02-17","cited_by_count":12,"area":null},{"id":"2104.14279","title":"How (Non-)Optimal is the Lexicon?","created":"2021-04-29","cited_by_count":11,"area":null},{"id":"2210.07095","title":"Incorporating Context into Subword Vocabularies","created":"2022-10-13","cited_by_count":11,"area":null},{"id":"2402.13432","title":"DrBenchmark: A Large Language Understanding Evaluation Benchmark for French Biomedical Domain","created":"2024-02-20","cited_by_count":11,"area":null},{"id":"2506.03101","title":"Beyond Text Compression: Evaluating Tokenizers Across Scales","created":"2025-06-03","cited_by_count":11,"area":null},{"id":"2601.07220","title":"The Roots of Performance Disparity in Multilingual Language Models: Intrinsic Modeling Difficulty or Design Choices?","created":"2026-01-12","cited_by_count":11,"area":null},{"id":"2304.09448","title":"EC^2: Emergent Communication for Embodied Control","created":"2023-04-19","cited_by_count":10,"area":null},{"id":"2307.07262","title":"MorphPiece : A Linguistic Tokenizer for Large Language Models","created":"2023-07-14","cited_by_count":9,"area":null},{"id":"2308.00683","title":"CodeBPE: Investigating Subtokenization Options for Large Language Model Pretraining on Source Code","created":"2023-08-01","cited_by_count":9,"area":null},{"id":"2310.15113","title":"Counting the Bugs in ChatGPT's Wugs: A Multilingual Investigation into the Morphological Capabilities of a Large Language Model","created":"2023-10-23","cited_by_count":9,"area":null},{"id":"2109.05772","title":"Wine is Not v i n. -- On the Compatibility of Tokenizations Across Languages","created":"2021-09-13","cited_by_count":8,"area":null},{"id":"2209.04126","title":"MaxMatch-Dropout: Subword Regularization for WordPiece","created":"2022-09-09","cited_by_count":8,"area":null},{"id":"2210.02956","title":"Are word boundaries useful for unsupervised language learning?","created":"2022-10-06","cited_by_count":8,"area":null},{"id":"2404.13292","title":"Evaluating Subword Tokenization: Alien Subword Composition and OOV Generalization Challenge","created":"2024-04-20","cited_by_count":8,"area":null},{"id":"2212.09912","title":"Tokenization Consistency Matters for Generative Models on Extractive NLP Tasks","created":"2022-12-19","cited_by_count":7,"area":null},{"id":"2309.12491","title":"Exploring the Impact of Training Data Distribution and Subword Tokenization on Gender Bias in Machine Translation","created":"2023-09-21","cited_by_count":7,"area":null},{"id":"2410.23684","title":"Improbable Bigrams Expose Vulnerabilities of Incomplete Tokens in Byte-Level Tokenizers","created":"2024-10-31","cited_by_count":7,"area":null},{"id":"2505.22645","title":"Characterizing Bias: Benchmarking Large Language Models in Simplified versus Traditional Chinese","created":"2025-05-28","cited_by_count":7,"area":null},{"id":"2507.21830","title":"DualSG: A Dual-Stream Explicit Semantic-Guided Multivariate Time Series Forecasting Framework","created":"2025-07-29","cited_by_count":7,"area":null},{"id":"2510.21909","title":"Explaining and Mitigating Crosslingual Tokenizer Inequities","created":"2025-10-24","cited_by_count":7,"area":null},{"id":"2305.05461","title":"What is the best recipe for character-level encoder-only modelling?","created":"2023-05-09","cited_by_count":6,"area":null},{"id":"2310.10260","title":"Prediction of Arabic Legal Rulings using Large Language Models","created":"2023-10-16","cited_by_count":6,"area":null},{"id":"2402.14616","title":"The Impact of Word Splitting on the Semantic Content of Contextualized Word Representations","created":"2024-02-22","cited_by_count":6,"area":null},{"id":"2406.07025","title":"Entropy-Reinforced Planning with Large Language Models for Drug Discovery","created":"2024-06-11","cited_by_count":6,"area":null},{"id":"2504.02122","title":"Overcoming Vocabulary Constraints with Pixel-level Fallback","created":"2025-04-02","cited_by_count":6,"area":null},{"id":"2601.13260","title":"Stop Taking Tokenizers for Granted: They Are Core Design Decisions in Large Language Models","created":"2026-01-19","cited_by_count":6,"area":null},{"id":"2205.11490","title":"Local Byte Fusion for Neural Machine Translation","created":"2022-05-23","cited_by_count":5,"area":null},{"id":"2312.02598","title":"Impact of Tokenization on LLaMa Russian Adaptation","created":"2023-12-05","cited_by_count":5,"area":null},{"id":"2401.18034","title":"Paramanu: Compact and Competitive Monolingual Language Models for Low-Resource Morphologically Rich Indian Languages","created":"2024-01-31","cited_by_count":5,"area":null},{"id":"2403.11130","title":"Exploring Tokenization Strategies and Vocabulary Sizes for Enhanced Arabic Language Models","created":"2024-03-17","cited_by_count":5,"area":null},{"id":"2405.00602","title":"Investigating Automatic Scoring and Feedback using Large Language Models","created":"2024-05-01","cited_by_count":5,"area":null},{"id":"2412.11940","title":"The Impact of Token Granularity on the Predictive Power of Language Model Surprisal","created":"2024-12-16","cited_by_count":5,"area":null},{"id":"2501.06246","title":"A Partition Cover Approach to Tokenization","created":"2025-01-08","cited_by_count":5,"area":null},{"id":"2606.27019","title":"MinGram: A Minimalist Unigram Tokenizer with High Compression and Competitive Morphological Alignment","created":"2026-06-25","cited_by_count":5,"area":null},{"id":"2306.07764","title":"Tokenization with Factorized Subword Encoding","created":"2023-06-13","cited_by_count":4,"area":null},{"id":"2309.03040","title":"Automated CVE Analysis for Threat Prioritization and Impact Prediction","created":"2023-09-06","cited_by_count":4,"area":null},{"id":"2412.12310","title":"Second Language (Arabic) Acquisition of LLMs via Progressive Vocabulary Expansion","created":"2024-12-16","cited_by_count":4,"area":null},{"id":"2503.17247","title":"KL3M Tokenizers: A Family of Domain-Specific and Character-Level Tokenizers for Legal, Financial, and Preprocessing Applications","created":"2025-03-21","cited_by_count":4,"area":null},{"id":"2510.05181","title":"Auditing Pay-Per-Token in Large Language Models","created":"2025-10-05","cited_by_count":4,"area":null},{"id":"2303.14234","title":"SIGMORPHON 2023 Shared Task of Interlinear Glossing: Baseline Model","created":"2023-03-24","cited_by_count":3,"area":null},{"id":"2409.06639","title":"TeXBLEU: Automatic Metric for Evaluate LaTeX Format","created":"2024-09-10","cited_by_count":3,"area":null},{"id":"2504.15120","title":"Kuwain 1.5B: An Arabic SLM via Language Injection","created":"2025-04-21","cited_by_count":3,"area":null},{"id":"2510.09947","title":"Beyond Fertility: Analyzing STRR as a Metric for Multilingual Tokenization Evaluation","created":"2025-10-11","cited_by_count":3,"area":null},{"id":"2511.05578","title":"UTF-8 Plumbing: Byte-level Tokenizers Unavoidably Enable LLMs to Generate Ill-formed UTF-8","created":"2025-11-05","cited_by_count":3,"area":null}],"next_cursor":"100"}