# Large Language Models are Pattern Matchers: Editing Semi-Structured and Structured Documents with ChatGPT

arXiv: https://arxiv.org/abs/2409.07732 · HTML: https://mlsanity.com/paper/2409.07732/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 24 references that are in the map.
- First posted on arXiv 12 Sep 2024.

## Builds on (most cited first, 15 of 15)

- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Instruction-Following Evaluation for Large Language Models](https://mlsanity.com/paper/2311.07911/) (2023)
- [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://mlsanity.com/paper/2111.01243/) (2021)
- [StructGPT: A General Framework for Large Language Model to Reason over Structured Data](https://mlsanity.com/paper/2305.09645/) (2023)
- [Language is All a Graph Needs](https://mlsanity.com/paper/2308.07134/) (2023)
- [Table Meets LLM: Can Large Language Models Understand Structured Table Data? A Benchmark and Empirical Study](https://mlsanity.com/paper/2305.13062/) (2023)
- [Language Models Enable Simple Systems for Generating Structured Views of Heterogeneous Data Lakes](https://mlsanity.com/paper/2304.09433/) (2023)
- [Coreference Resolution as Query-based Span Prediction](https://mlsanity.com/paper/1911.01746/) (2019)
- [Extracting Accurate Materials Data from Research Papers with Conversational Language Models and Prompt Engineering](https://mlsanity.com/paper/2303.05352/) (2023)
- [FOFO: A Benchmark to Evaluate LLMs' Format-Following Capability](https://mlsanity.com/paper/2402.18667/) (2024)
- [Label-free Node Classification on Graphs with Large Language Models (LLMS)](https://mlsanity.com/paper/2310.04668/) (2023)
- [Tabular Representation, Noisy Operators, and Impacts on Table Structure Understanding Tasks in LLMs](https://mlsanity.com/paper/2310.10358/) (2023)
- [Beyond the Chat: Executable and Verifiable Text-Editing with LLMs](https://mlsanity.com/paper/2309.15337/) (2023)
- [Large Language Models as Software Components: A Taxonomy for LLM-Integrated Applications](https://mlsanity.com/paper/2406.10300/) (2024)
- [Development of a multi-timescale method for classifying hybrid energy storage systems in grid applications](https://mlsanity.com/paper/2203.07750/) (2022)

## Built on by (most cited first, 2 of 2)

- [SV-TrustEval-C: Evaluating Structure and Semantic Reasoning in Large Language Models for Source Code Vulnerability Analysis](https://mlsanity.com/paper/2505.20630/) (2025)
- [Gaze-to-text Generation: Beyond Categorical Decoding of Human Attention](https://mlsanity.com/paper/2607.23917/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) offer numerous applications, the full extent of which is not yet understood. This paper investigates if LLMs can be applied for editing structured and semi-structured documents with minimal effort. Using a qualitative research approach, we conduct two case studies with ChatGPT and thoroughly analyze the results. Our experiments indicate that LLMs can effectively edit structured and semi-structured documents when provided with basic, straightforward prompts. ChatGPT demonstrates a strong ability to recognize and process the structure of annotated documents. This suggests that explicitly structuring tasks and data in prompts might enhance an LLM's ability to understand and solve tasks. Furthermore, the experiments also reveal impressive pattern matching skills in ChatGPT. This observation deserves further investigation, as it may contribute to understanding the processes leading to hallucinations in LLMs.