mlsanityData to 14 Oct 2026

Augmented Language Models: a Survey

arXiv 2302.07842 ↗First posted 15 Feb 2023cs.CLcited by 403

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

This survey reviews works in which language models (LMs) are augmented with reasoning skills and the ability to use tools. The former is defined as decomposing a potentially complex task into simpler subtasks while the latter consists in calling external modules such as a code interpreter. LMs can leverage these augmentations separately or in combination via heuristics, or learn to do so from demonstrations. While adhering to a standard missing tokens prediction objective, such augmented LMs can use various, possibly non-parametric external modules to expand their context processing ability, thus departing from the pure language modeling paradigm. We therefore refer to them as Augmented Language Models (ALMs). The missing token objective allows ALMs to learn to reason, use tools, and even act, while still performing standard natural language tasks and even outperforming most regular LMs on several benchmarks. In this work, after reviewing current advance in ALMs, we conclude that this new research direction has the potential to address common limitations of traditional LMs such as interpretability, consistency, and scalability issues.

Key facts. Cited by 403 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 154 of its 191 references that are in the map. First posted on arXiv 15 Feb 2023.

Builds on

154 of 191 references.

Closest ancestors · 14

Same direction, newest first

  1. 2022Rethinking with Retrieval: Faithful Large Language Model Inference31 Dec
  2. 2022LAMBADA: Backward Chaining for Automated Reasoning in Natural Language20 Dec
  3. 2022Towards Reasoning in Large Language Models: A Survey20 Dec
  4. 2022ALERT: Adapting Language Models to Reasoning Tasks16 Dec
  5. 2022PAL: Program-aided Language Models18 Nov
  6. 2022Language Models of Code are Few-Shot Commonsense Learners13 Oct
  7. 2022Mind's Eye: Grounded Language Model Reasoning through Simulation11 Oct
  8. 2022Measuring and Narrowing the Compositionality Gap in Language Models7 Oct
  9. 2022ReAct: Synergizing Reasoning and Acting in Language Models6 Oct
  10. 2022Faithful Reasoning Using Large Language Models30 Aug
  11. 2022Language models show human-like content effects on reasoning tasks14 Jul
  12. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning19 May
2 more
  1. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?25 Feb
  2. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Models30 Nov

Other references · 140

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2017Proximal Policy Optimization Algorithmscited by 15,215
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  7. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  8. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  9. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  10. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  11. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  12. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
38 more
  1. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  2. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  4. 2017Deep reinforcement learning from human preferencescited by 3,912
  5. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  6. 2016Pointer Sentinel Mixture Modelscited by 3,096
  7. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  8. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  9. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  10. 2020Learning to summarize from human feedbackcited by 2,588
  11. 2022Emergent Abilities of Large Language Modelscited by 2,512
  12. 2017Billion-scale similarity search with GPUscited by 2,497
  13. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  14. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  15. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  16. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  17. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  18. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  19. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  20. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
  21. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  22. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  23. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  24. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  25. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,407
  26. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,392
  27. 2021Improving language models by retrieving from trillions of tokenscited by 1,293
  28. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,251
  29. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
  30. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,210
  31. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,210
  32. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,202
  33. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,135
  34. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,081
  35. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,076
  36. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,035
  37. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,027
  38. 2019How Can We Know What Language Models Know?cited by 959

50 most cited of 140 All 140 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 161; 2024: 109; 2025: 74; 2026: 59.

403 papers.

Sort:

By area:

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  2. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  3. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  4. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,242
  5. 2023Reasoning with Language Model is Planning with World Modelcited by 861
  6. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 853
  7. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 783
  8. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 708
  9. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 688
  10. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 681
  11. 2023A Comprehensive Overview of Large Language Modelscited by 648
  12. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 634
38 more
  1. 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMscited by 591
  2. 2024Executable Code Actions Elicit Better LLM Agentscited by 564
  3. 2024Large Language Models: A Surveycited by 525
  4. 2023Cognitive Architectures for Language Agentscited by 429
  5. 2023Language Models can Solve Computer Taskscited by 372
  6. 2023Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergycited by 346
  7. 2024AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agentscited by 339
  8. 2024Benchmarking Retrieval-Augmented Generation for Medicinecited by 331
  9. 2024Improving Alignment and Robustness with Circuit Breakerscited by 322
  10. 2023Tool Learning with Foundation Modelscited by 302
  11. 2023MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedbackcited by 297
  12. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 284
  13. 2023ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solvingcited by 282
  14. 2024The Power of Noise: Redefining Retrieval for RAG Systemscited by 281
  15. 2023Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Modelscited by 264
  16. 2023Challenges and Applications of Large Language Modelscited by 254
  17. 2023Large Language Model Alignment: A Surveycited by 243
  18. 2023ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddingscited by 240
  19. 2023PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimizationcited by 233
  20. 2024RepairAgent: An Autonomous, LLM-Based Agent for Program Repaircited by 232
  21. 2024Tool Learning with Large Language Models: A Surveycited by 223
  22. 2023Can Pre-trained Vision and Language Models Answer Visual Information-Seeking Questions?cited by 207
  23. 2024OS-Copilot: Towards Generalist Computer Agents with Self-Improvementcited by 205
  24. 2023Evaluating Large Language Models: A Comprehensive Surveycited by 202
  25. 2023OpenAGI: When LLM Meets Domain Expertscited by 197
  26. 2023Synapse: Trajectory-as-Exemplar Prompting with Memory for Computer Controlcited by 187
  27. 2023MedCPT: Contrastive Pre-trained Transformers with Large-scale PubMed Search Logs for Zero-shot Biomedical Information Retrievalcited by 185
  28. 2023TaskMatrix.AI: Completing Tasks by Connecting Foundation Models with Millions of APIscited by 184
  29. 2023ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Modelscited by 184
  30. 2024Large Language Model-Based Agents for Software Engineering: A Surveycited by 178
  31. 2023Can LLM-Generated Misinformation Be Detected?cited by 177
  32. 2023GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Modelscited by 176
  33. 2023Image Hijacks: Adversarial Images can Control Generative Models at Runtimecited by 172
  34. 2023JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Modelscited by 171
  35. 2024The Effect of Sampling Temperature on Problem Solving in Large Language Modelscited by 170
  36. 2023LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic Proverscited by 168
  37. 2025Prompt Injection Attack to Tool Selection in LLM Agentscited by 158
  38. 2024Continual Learning for Large Language Models: A Surveycited by 147

50 most cited of 403 All 403 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024LLM Augmented LLMs: Expanding Capabilities through Composition
  2. 2025Fast, Slow, and Tool-augmented Thinking for LLMs: A Review
  3. 2025Benchmarking Failures in Tool-Augmented Language Models
  4. 2024Can Tool-augmented Large Language Models be Aware of Incomplete Conditions?
  5. 2024Language hooks: a modular framework for augmenting LLM reasoning that decouples tool usage from the model and its prompt
  6. 2024LLM With Tools: A Survey
  7. 2025Thinking Isn't an Illusion: Overcoming the Limitations of Reasoning Models via Tool Augmentations
  8. 2025On the Notion that Language Models Reason