mlsanityData to 7 Oct 2026

Adam's Law: Textual Frequency Law on Large Language Models

arXiv 2604.02176 ↗First posted 2 Apr 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

While textual frequency has been validated as relevant to human cognition in reading speed, its relatedness to Large Language Models (LLMs) is seldom studied. We propose a novel research direction in terms of textual data frequency, which is an understudied topic, to the best of our knowledge. Our framework is composed of three units. First, this paper proposes Textual Frequency Law (TFL), which indicates that frequent textual data should be preferred for LLMs for both prompting and fine-tuning. Since many LLMs are closed-source in their training data, we propose using online resources to estimate the sentence-level frequency. We then utilize an input paraphraser to paraphrase the input into a more frequent textual expression. Next, we propose Textual Frequency Distillation (TFD) by querying LLMs to conduct story completion by further extending the sentences in the datasets, and the resulting corpora are used to adjust the initial estimation. Finally, we propose Curriculum Textual Frequency Training (CTFT) that fine-tunes LLMs in an increasing order of sentence-level frequency. Experiments are conducted on our curated dataset Textual Frequency Paired Dataset (TFPD) on math reasoning, machine translation, commonsense reasoning and agentic tool calling. Results show the effectiveness of our framework.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 15 of its 38 references that are in the map. First posted on arXiv 2 Apr 2026.

Builds on

15 of 38 references.

References · 15

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  3. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  4. 2020COMET: A Neural Framework for MT Evaluationcited by 1,024
  5. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 789
  6. 2022Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matterscited by 221
  7. 2020Neural Syntactic Preordering for Controlled Paraphrase Generationcited by 56
  8. 2020Paraphrase Augmented Task-Oriented Dialog Generationcited by 43
  9. 2024Frequency Explains the Inverse Correlation of Large Language Models' Size, Training Data Amount, and Surprisal's Fit to Reading Timescited by 37
  10. 2024Quality Matters: Evaluating Synthetic Data for Tool-Using LLMscited by 17
  11. 2023LM-CPPF: Paraphrasing-Guided Data Augmentation for Contrastive Prompt-Based Few-Shot Fine-Tuningcited by 15
  12. 2022PCC: Paraphrasing with Bottom-k Sampling and Cyclic Learning for Curriculum Data Augmentationcited by 14
3 more
  1. 2023Not All Metrics Are Guilty: Improving NLG Evaluation by Diversifying Referencescited by 11
  2. 2023Transformer Language Models Handle Word Frequency in Prediction Headcited by 8
  3. 2025Generalizing From Short to Long: Effective Data Synthesis for Long-Context Instruction Tuningcited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Large Linguistic Models: Investigating LLMs' metalinguistic abilities
  2. 2024Sample-Efficient Human Evaluation of Large Language Models via Maximum Discrepancy Competition
  3. 2023Are Large Language Models Temporally Grounded?
  4. 2023Psychometric Predictive Power of Large Language Models
  5. 2025Thinking Augmented Pre-training
  6. 2023To Repeat or Not To Repeat: Insights from Scaling LLM under Token-Crisis
  7. 2024Sloth: scaling laws for LLM skills to predict multi-benchmark performance across families
  8. 2024A critical review of methods and challenges in large language models