mlsanityData to 7 Oct 2026

Binding Language Models in Symbolic Languages

arXiv 2210.02875 ↗First posted 6 Oct 2022cs.CLcited by 224

Not on the map. Why?

Cited by 81 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Though end-to-end neural approaches have recently been dominating NLP tasks in both performance and ease-of-use, they lack interpretability and robustness. We propose Binder, a training-free neural-symbolic framework that maps the task input to a program, which (1) allows binding a unified API of language model (LM) functionalities to a programming language (e.g., SQL, Python) to extend its grammar coverage and thus tackle more diverse questions, (2) adopts an LM as both the program parser and the underlying model called by the API during execution, and (3) requires only a few in-context exemplar annotations. Specifically, we employ GPT-3 Codex as the LM. In the parsing stage, with only a few in-context exemplars, Codex is able to identify the part of the task input that cannot be answerable by the original programming language, correctly generate API calls to prompt Codex to solve the unanswerable part, and identify where to place the API calls while being compatible with the original grammar. In the execution stage, Codex can perform versatile functionalities (e.g., commonsense QA, information extraction) given proper prompts in the API calls. Binder achieves state-of-the-art results on WikiTableQuestions and TabFact datasets, with explicit output programs that benefit human debugging. Note that previous best systems are all finetuned on tens of thousands of task-specific samples, while Binder only uses dozens of annotations as in-context exemplars without any training. Our code is available at https://github.com/HKUNLP/Binder .

Key facts. Cited by 224 papers in the map (data through 7 Oct 2026). Builds on 48 of its 67 references that are in the map. First posted on arXiv 6 Oct 2022.

Builds on

48 of 67 references.

References · 48

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  7. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  9. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  10. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  11. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  12. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 679
36 more
  1. 2015Compositional Semantic Parsing on Semi-Structured Tablescited by 509
  2. 2020TAPAS: Weakly Supervised Table Parsing via Pre-trainingcited by 432
  3. 2019TabFact: A Large-scale Dataset for Table-based Fact Verificationcited by 425
  4. 2021PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Modelscited by 402
  5. 2020TaBERT: Pretraining for Joint Understanding of Textual and Tabular Datacited by 395
  6. 2015Neural Module Networkscited by 375
  7. 2022CodeT: Code Generation with Generated Testscited by 369
  8. 2022Selective Annotation Makes Language Models Better Few-Shot Learnerscited by 242
  9. 2021TAPEX: Table Pre-training via Learning a Neural SQL Executorcited by 233
  10. 2022UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Modelscited by 231
  11. 2021MultiModalQA: Complex Question Answering over Text, Tables and Imagescited by 177
  12. 2020GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsingcited by 171
  13. 2020Compositional Generalization and Natural Language Variation: Can a Semantic Parsing Approach Handle Both?cited by 147
  14. 2020Break It Down: A Question Understanding Benchmarkcited by 131
  15. 2017Learning to Reason: End-to-End Module Networks for Visual Question Answeringcited by 125
  16. 2022Natural Language to Code Translation with Executioncited by 122
  17. 2016Neural Symbolic Machines: Learning Semantic Parsers on Freebase with Weak Supervisioncited by 101
  18. 2022Evaluating the Text-to-SQL Capabilities of Large Language Modelscited by 101
  19. 2020Understanding tables with intermediate pre-trainingcited by 68
  20. 2019Neural Module Networks for Reasoning over Textcited by 65
  21. 2020Text Modular Networks: Learning to Decompose Tasks in the Language of Existing Modelscited by 65
  22. 2022Neural-Symbolic Models for Logical Queries on Knowledge Graphscited by 60
  23. 2020On the Potential of Lexico-logical Alignments for Semantic Parsing to SQL Queriescited by 59
  24. 2018Memory Augmented Policy Optimization for Program Synthesis and Semantic Parsingcited by 57
  25. 2018Neural Modular Control for Embodied Question Answeringcited by 56
  26. 2022In-Context Learning for Few-Shot Dialogue State Trackingcited by 47
  27. 2019Learning to Generalize from Sparse and Underspecified Rewardscited by 39
  28. 2021Turning Tables: Generating Examples from Semi-structured Tables for Endowing Language Models with Reasoning Skillscited by 39
  29. 2015Neural Programmer: Inducing Latent Programs with Gradient Descentcited by 32
  30. 2020Program Enhanced Fact Verification with Verbalization and Graph Attention Networkcited by 29
  31. 2022OmniTab: Pretraining with Natural and Synthetic Data for Few-shot Table-based Question Answeringcited by 29
  32. 2019Learning Semantic Parsers from Denotations with Latent Structured Alignments and Abstract Programscited by 26
  33. 2020LogicalFactChecker: Leveraging Logical Operations for Fact Checking with Graph Module Networkcited by 26
  34. 2021Database Reasoning Over Textcited by 20
  35. 2022TaCube: Pre-computing Data Cubes for Answering Numerical-Reasoning Questions over Tabular Datacited by 19
  36. 2017Macro Grammars and Holistic Triggering for Efficient Semantic Parsingcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 13; 2023: 60; 2024: 74; 2025: 47; 2026: 30.

224 papers.

Sort:

By area:

  1. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  2. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  3. 2022PAL: Program-aided Language Modelscited by 635
  4. 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 591
  5. 2023StructGPT: A General Framework for Large Language Model to Reason over Structured Datacited by 339
  6. 2022Reasoning with Language Model Prompting: A Surveycited by 251
  7. 2023LEVER: Learning to Verify Language-to-Code Generation with Executioncited by 228
  8. 2024Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understandingcited by 185
  9. 2023Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Futurecited by 184
  10. 2023Large Language Models are Versatile Decomposers: Decompose Evidence and Questions for Table-based Reasoningcited by 177
  11. 2023LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic Proverscited by 161
  12. 2022DePlot: One-shot visual language reasoning by plot-to-table translationcited by 157
38 more
  1. 2024Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Surveycited by 157
  2. 2023Fact-Checking Complex Claims with Program-Guided Reasoningcited by 146
  3. 2023Large Language Model Cascades with Mixture of Thoughts Representations for Cost-efficient Reasoningcited by 140
  4. 2023ChatDB: Augmenting LLMs with Databases as Their Symbolic Memorycited by 136
  5. 2023OpenAgents: An Open Platform for Language Agents in the Wildcited by 136
  6. 2023Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Surveycited by 134
  7. 2022MatCha: Enhancing Visual Language Pretraining with Math Reasoning and Chart Derenderingcited by 132
  8. 2024Debug like a Human: A Large Language Model Debugger via Verifying Runtime Execution Step-by-stepcited by 123
  9. 2023A comprehensive evaluation of ChatGPT's zero-shot Text-to-SQL capabilitycited by 118
  10. 2022Large Language Models are few(1)-shot Table Reasonerscited by 116
  11. 2023Large Language Models as Analogical Reasonerscited by 107
  12. 2022PromptCap: Prompt-Guided Task-Aware Image Captioningcited by 104
  13. 2024Large Language Models Meet NLP: A Surveycited by 102
  14. 2024SpreadsheetBench: Towards Challenging Real World Spreadsheet Manipulationcited by 100
  15. 2023Can Knowledge Graphs Reduce Hallucinations in LLMs? : A Surveycited by 92
  16. 2024Large Language Models and Games: A Survey and Roadmapcited by 91
  17. 2022Natural Language to Code Generation in Interactive Data Science Notebookscited by 86
  18. 2023Batch Prompting: Efficient Inference with Large Language Model APIscited by 84
  19. 2023Retrieving Multimodal Information for Augmented Generation: A Surveycited by 84
  20. 2024AgentGen: Enhancing Planning Abilities for Large Language Model based Agent via Environment and Task Generationcited by 80
  21. 2025Lifelong Learning of Large Language Model based Agents: A Roadmapcited by 78
  22. 2024If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agentscited by 77
  23. 2024TableLLM: Enabling Tabular Data Manipulation by LLMs in Real Office Usage Scenarioscited by 76
  24. 2023Few-shot In-context Learning for Knowledge Base Question Answeringcited by 74
  25. 2023Lemur: Harmonizing Natural Language and Code for Language Agentscited by 71
  26. 2024TroVE: Inducing Verifiable and Efficient Toolboxes for Solving Programmatic Taskscited by 70
  27. 2023Applying Large Language Models and Chain-of-Thought for Automatic Scoringcited by 68
  28. 2023CodeIE: Large Code Generation Models are Better Few-Shot Information Extractorscited by 67
  29. 2024A Survey on Employing Large Language Models for Text-to-SQL Taskscited by 67
  30. 2023Visual Program Distillation: Distilling Tools and Programmatic Reasoning into Vision-Language Modelscited by 66
  31. 2023Scallop: A Language for Neurosymbolic Programmingcited by 63
  32. 2022Parsel: Algorithmic Reasoning with Language Models by Composing Decompositionscited by 62
  33. 2023Clover: Closed-Loop Verifiable Code Generationcited by 62
  34. 2022Don't Generate, Discriminate: A Proposal for Grounding Language Models to Real-World Environmentscited by 57
  35. 2023How to Prompt LLMs for Text-to-SQL: A Study in Zero-shot, Single-domain, and Cross-domain Settingscited by 57
  36. 2024DoraemonGPT: Toward Understanding Dynamic Scenes with Large Language Models (Exemplified as A Video Agent)cited by 56
  37. 2023EHRXQA: A Multi-Modal Question Answering Dataset for Electronic Health Records with Chest X-ray Imagescited by 53
  38. 2024What Are Tools Anyway? A Survey from the Language Model Perspectivecited by 53

50 most cited of 224 All 224 (JSON)