# LLM-Generated Natural Language Meets Scaling Laws: New Explorations and Data Augmentation Methods

arXiv: https://arxiv.org/abs/2407.00322 · HTML: https://mlsanity.com/paper/2407.00322/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 51 references that are in the map.
- First posted on arXiv 29 Jun 2024.

## Builds on (most cited first, 26 of 26)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks](https://mlsanity.com/paper/1901.11196/) (2019)
- [How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection](https://mlsanity.com/paper/2301.07597/) (2023)
- [Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond](https://mlsanity.com/paper/2304.13712/) (2023)
- [ChatGPT: Jack of all trades, master of none](https://mlsanity.com/paper/2302.10724/) (2023)
- [AugGPT: Leveraging ChatGPT for Text Data Augmentation](https://mlsanity.com/paper/2302.13007/) (2023)
- [Tuning Language Models as Training Data Generators for Augmentation-Enhanced Few-Shot Learning](https://mlsanity.com/paper/2211.03044/) (2022)
- [Zero-Shot Text Classification with Self-Training](https://mlsanity.com/paper/2210.17541/) (2022)
- [Evaluating Computational Language Models with Scaling Properties of Natural Language](https://mlsanity.com/paper/1906.09379/) (2019)
- [Complex systems approach to natural language](https://mlsanity.com/paper/2401.02772/) (2024)
- [Improving Short Text Classification With Augmented Data Using GPT-3](https://mlsanity.com/paper/2205.10981/) (2022)
- [Multiperiodic Processes: Ergodic Sources with a Sublinear Entropy](https://mlsanity.com/paper/2302.09049/) (2023)
- [Natural Language Statistical Features of LSTM-generated Texts](https://mlsanity.com/paper/1804.04087/) (2018)
- [A novel knowledge graph development for industry design: A case study on indirect coal liquefaction process](https://mlsanity.com/paper/2111.13854/) (2021)
- [Construction contract risk identification based on knowledge-augmented language model](https://mlsanity.com/paper/2309.12626/) (2023)
- [Semi-Supervised Heterogeneous Graph Learning with Multi-level Data Augmentation](https://mlsanity.com/paper/2212.00024/) (2022)
- [Benford's law: a 'sleeping beauty' sleeping in the dirty pages of logarithmic tables](https://mlsanity.com/paper/1702.00554/) (2017)
- [A New Multifractal-based Deep Learning Model for Text Mining](https://mlsanity.com/paper/2111.13861/) (2021)
- [A Zipf's Law-based Text Generation Approach for Addressing Imbalance in Entity Extraction](https://mlsanity.com/paper/2205.12636/) (2022)
- [A new hazard event classification model via deep learning and multifractal](https://mlsanity.com/paper/2209.05263/) (2022)
- [WC-SBERT: Zero-Shot Text Classification via SBERT with Self-Training for Wikipedia Categories](https://mlsanity.com/paper/2307.15293/) (2023)
- [Benford's law: what does it say on adversarial images?](https://mlsanity.com/paper/2102.04615/) (2021)
- [Will sentiment analysis need subculture? A new data augmentation approach](https://mlsanity.com/paper/2309.00178/) (2023)
- [Exploring the Law of Numbers: Evidence from China's Real Estate](https://mlsanity.com/paper/2309.05221/) (2023)

## Built on by (most cited first, 8 of 8)

- [GPIoT: Tailoring Small Language Models for IoT Program Synthesis and Development](https://mlsanity.com/paper/2503.00686/) (2025)
- [Scaling-up Perceptual Video Quality Assessment](https://mlsanity.com/paper/2505.22543/) (2025)
- [Evaluating and Enhancing LLMs for Multi-turn Text-to-SQL with Multiple Question Types](https://mlsanity.com/paper/2412.17867/) (2024)
- [ChemActor: Enhancing Automated Extraction of Chemical Synthesis Actions with LLM-Generated Data](https://mlsanity.com/paper/2506.23520/) (2025)
- [The Surprising Universality of LLM Outputs: A Real-Time Verification Primitive](https://mlsanity.com/paper/2604.25634/) (2026)
- [GOBench: Benchmarking Geometric Optics Generation and Understanding of MLLMs](https://mlsanity.com/paper/2506.00991/) (2025)
- [Peak-Detector: Explainable Peak Detection via Instruction-Tuned Large Language Models in Physiological Sign](https://mlsanity.com/paper/2605.16452/) (2026)
- [How Do LLM-Generated Texts Impact Term-Based Retrieval Models?](https://mlsanity.com/paper/2508.17715/) (2025)

## Abstract (arXiv)

With the ascent of large language models (LLM), natural language processing has witnessed enhancements, such as LLM-based data augmentation. Nonetheless, prior research harbors two primary concerns: firstly, a lack of contemplation regarding whether the natural language generated by LLM (LLMNL) truly aligns with human natural language (HNL), a critical foundational question; secondly, an oversight that augmented data is randomly generated by LLM, implying that not all data may possess equal training value, that could impede the performance of classifiers. To address these challenges, we introduce the scaling laws to intrinsically calculate LLMNL and HNL. Through extensive experiments, we reveal slight deviations (approximately 0.2 Mandelbrot exponent) from Mandelbrot's law in LLMNL, underscore a complexity advantage in HNL, and supplement an interpretive discussion on language style. This establishes a solid foundation for LLM's expansion. Further, we introduce a novel data augmentation method for few-shot text classification, termed ZGPTDA, which leverages fuzzy computing mechanisms driven by the conformity to scaling laws to make decisions about GPT-4 augmented data. Extensive experiments, conducted in real-world scenarios, confirms the effectiveness (improving F1 of Bert and RoBerta by 7-10%) and competitiveness (surpassing recent AugGPT and GENCO methods by about 2% accuracy on DeBerta) of ZGPTDA. In addition, we reveal some interesting insights, e.g., Hilberg's law and Taylor's law can impart more benefits to text classification, etc.