mlsanityData to 7 Oct 2026

Large Language Models as General Pattern Machines

arXiv 2307.04721 ↗First posted 10 Jul 2023cs.AI cs.CL cs.ROcited by 196

Not on the map. Why?

Cited by 31 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We observe that pre-trained large language models (LLMs) are capable of autoregressively completing complex token sequences -- from arbitrary ones procedurally generated by probabilistic context-free grammars (PCFG), to more rich spatial patterns found in the Abstraction and Reasoning Corpus (ARC), a general AI benchmark, prompted in the style of ASCII art. Surprisingly, pattern completion proficiency can be partially retained even when the sequences are expressed using tokens randomly sampled from the vocabulary. These results suggest that without any additional training, LLMs can serve as general sequence modelers, driven by in-context learning. In this work, we investigate how these zero-shot capabilities may be applied to problems in robotics -- from extrapolating sequences of numbers that represent states over time to complete simple motions, to least-to-most prompting of reward-conditioned trajectories that can discover and represent closed-loop policies (e.g., a stabilizing controller for CartPole). While difficult to deploy today for real systems due to latency, context size limitations, and compute costs, the approach of using LLMs to drive low-level control may provide an exciting glimpse into how the patterns among words could be transferred to actions.

Key facts. Cited by 196 papers in the map (data through 7 Oct 2026). Builds on 68 of its 94 references that are in the map. First posted on arXiv 10 Jul 2023.

Builds on

68 of 94 references.

References · 68

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  8. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  9. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  10. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  11. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  12. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
38 more
  1. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  2. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  3. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  4. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  5. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  6. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  7. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  8. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  9. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  10. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  11. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  12. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  13. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  14. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
  15. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  16. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  17. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
  18. 2022Transformers learn in-context by gradient descentcited by 714
  19. 2022What Can Transformers Learn In-Context? A Case Study of Simple Function Classescited by 700
  20. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  21. 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 671
  22. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 598
  23. 2022Socratic Models: Composing Zero-Shot Multimodal Reasoning with Languagecited by 513
  24. 2023LLM+P: Empowering Large Language Models with Optimal Planning Proficiencycited by 496
  25. 2022LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Modelscited by 459
  26. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoningcited by 331
  27. 2022Data Distributional Properties Drive Emergent In-Context Learning in Transformerscited by 303
  28. 2019Q8BERT: Quantized 8Bit BERTcited by 296
  29. 2020Pretrained Transformers Improve Out-of-Distribution Robustnesscited by 293
  30. 2019Compositionality decomposed: how do neural networks generalise?cited by 264
  31. 2023Reward Design with Language Modelscited by 257
  32. 2022Real-World Robot Learning with Masked Visual Pre-trainingcited by 241
  33. 2022Emergent Analogical Reasoning in Large Language Modelscited by 239
  34. 2023Text2Motion: From Natural Language Instructions to Feasible Planscited by 223
  35. 2022Pre-Trained Language Models for Interactive Decision-Makingcited by 220
  36. 2023TidyBot: Personalized Robot Assistance with Large Language Modelscited by 219
  37. 2022In-context Reinforcement Learning with Algorithm Distillationcited by 186
  38. 2023Language-Driven Representation Learning for Roboticscited by 163

50 most cited of 68 All 68 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 35; 2024: 97; 2025: 46; 2026: 18.

196 papers.

Sort:

By area:

  1. 2023Large Language Models as Optimizerscited by 830
  2. 2023ExpeL: LLM Agents Are Experiential Learnerscited by 814
  3. 2024Chronos: Learning the Language of Time Seriescited by 712
  4. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 680
  5. 2024The Platonic Representation Hypothesiscited by 362
  6. 2023Language Modeling Is Compressioncited by 225
  7. 2024PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMscited by 187
  8. 2023OneLLM: One Framework to Align All Modalities with Languagecited by 185
  9. 2024Agent AI: Surveying the Horizons of Multimodal Interactioncited by 167
  10. 2023TEST: Text Prototype Aligned Embedding to Activate LLM's Ability for Time Seriescited by 165
  11. 2024Large Language Models to Enhance Bayesian Optimizationcited by 127
  12. 2023Chain of Code: Reasoning with a Language Model-Augmented Code Emulatorcited by 120
38 more
  1. 2023Hypothesis Search: Inductive Reasoning with Language Modelscited by 119
  2. 2024Large Language Models for Time Series: A Surveycited by 113
  3. 2023Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlookcited by 109
  4. 2023Avalon's Game of Thoughts: Battle Against Deception through Recursive Contemplationcited by 102
  5. 2024Fine-Tuned Language Models Generate Stable Inorganic Materials as Textcited by 92
  6. 2023Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinementcited by 89
  7. 2024Improving Text-to-Image Consistency via Automatic Prompt Optimizationcited by 88
  8. 2023MindAgent: Emergent Gaming Interactioncited by 87
  9. 2024RAP: Retrieval-Augmented Planning with Contextual Memory for Multimodal LLM Agentscited by 80
  10. 2023AntGPT: Can Large Language Models Help Long-term Action Anticipation from Videos?cited by 79
  11. 2024Open-Endedness is Essential for Artificial Superhuman Intelligencecited by 79
  12. 2024In-Context Imitation Learning via Next-Token Predictioncited by 65
  13. 2023Penetrative AI: Making LLMs Comprehend the Physical Worldcited by 64
  14. 2024LLM3:Large Language Model-based Task and Motion Planning with Motion Failure Reasoningcited by 63
  15. 2024Real-World Robot Applications of Foundation Models: A Reviewcited by 60
  16. 2023Language Models as Zero-Shot Trajectory Generatorscited by 59
  17. 2023Experiential Co-Learning of Software-Developing Agentscited by 57
  18. 2024From Words to Numbers: Your Large Language Model Is Secretly A Capable Regressor When Given In-Context Examplescited by 55
  19. 2023Comparing Humans, GPT-4, and GPT-4V On Abstraction and Reasoning Taskscited by 54
  20. 2023Curated LLM: Synergy of LLMs and Data Curation for tabular augmentation in low-data regimescited by 52
  21. 2023Action Scene Graphs for Long-Form Understanding of Egocentric Videoscited by 51
  22. 2023Multimodal Neurons in Pretrained Text-Only Transformerscited by 49
  23. 2024Position: What Can Large Language Models Tell Us about Time Series Analysiscited by 47
  24. 2023Prompt a Robot to Walk with Large Language Modelscited by 44
  25. 2023Distilling and Retrieving Generalizable Knowledge for Robot Manipulation via Language Correctionscited by 44
  26. 2023Vamos: Versatile Action Models for Video Understandingcited by 40
  27. 2024Kinetix: Investigating the Training of General Agents through Open-Ended Physics-Based Control Taskscited by 40
  28. 2025Automated Hypothesis Validation with Agentic Sequential Falsificationscited by 40
  29. 2025RoboFAC: A Comprehensive Framework for Robotic Failure Analysis and Correctioncited by 37
  30. 2024PREGO: online mistake detection in PRocedural EGOcentric videoscited by 36
  31. 2024R+X: Retrieval and Execution from Everyday Human Videoscited by 36
  32. 2025LLMs are Greedy Agents: Effects of RL Fine-tuning on Decision-Making Abilitiescited by 36
  33. 2025EXP-Bench: Can AI Conduct AI Research Experiments?cited by 36
  34. 2025Self-Improving Language Models for Evolutionary Program Synthesis: A Case Study on ARC-AGIcited by 34
  35. 2024EVOLvE: Evaluating and Optimizing LLMs For In-Context Explorationcited by 33
  36. 2025TimeCAP: Learning to Contextualize, Augment, and Predict Time Series Events with Large Language Model Agentscited by 33
  37. 2024Reasoning Abilities of Large Language Models: In-Depth Analysis on the Abstraction and Reasoning Corpuscited by 32
  38. 2024LMAct: A Benchmark for In-Context Imitation Learning with Long Multimodal Demonstrationscited by 32

50 most cited of 196 All 196 (JSON)