Statistical Analysis of Sentence Structures through ASCII, Lexical Alignment and PCA
Not on the map. Why?
Abstract (arXiv)
While utilizing syntactic tools such as parts-of-speech (POS) tagging has helped us understand sentence structures and their distribution across diverse corpora, it is quite complex and poses a challenge in natural language processing (NLP). This study focuses on understanding sentence structure balance - usages of nouns, verbs, determiners, etc - harmoniously without relying on such tools. It proposes a novel statistical method that uses American Standard Code for Information Interchange (ASCII) codes to represent text of 11 text corpora from various sources and their lexical category alignment after using their compressed versions through PCA, and analyzes the results through histograms and normality tests such as Shapiro-Wilk and Anderson-Darling Tests. By focusing on ASCII codes, this approach simplifies text processing, although not replacing any syntactic tools but complementing them by offering it as a resource-efficient tool for assessing text balance. The story generated by Grok shows near normality indicating balanced sentence structures in LLM outputs, whereas 4 out of the remaining 10 pass the normality tests. Further research could explore potential applications in text quality evaluation and style analysis with syntactic integration for more broader tasks.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 5 of its 14 references that are in the map. First posted on arXiv 13 Mar 2025.
Builds on
5 of 14 references.
References · 5
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
- 2014A Tutorial on Principal Component Analysiscited by 139
- 2024Pillars of Grammatical Error Correction: Comprehensive Inspection Of Contemporary Approaches In The Era of Large Language Modelscited by 8
- 2024POS-tagging to highlight the skeletal structure of sentencescited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Text-Based Approaches to Item Alignment to Content Standards in Large-Scale Reading & Writing Tests
- 2026XQDT: eXplainable and Quantitative Data-Text Alignment Metric with Feedback Signals
- 2025Scaling Item-to-Standard Alignment with Large Language Models: Accuracy, Limits, and Solutions
- 2026More Aligned, Less Diverse? Analyzing the Grammar and Lexicon of Two Generations of LLMs
- 2023Text-To-KG Alignment: Comparing Current Methods on Classification Tasks
- 2025Fine-Tuned Language Models for Domain-Specific Summarization and Tagging
- 2021Word Alignment by Fine-tuning Embeddings on Parallel Corpora
- 2020Aligning the Pretraining and Finetuning Objectives of Language Models