mlsanityData to 14 Oct 2026

Neurosymbolic Routing for Reliable Reasoning on Resource-Constrained Edge Devices

arXiv 2609.35833 ↗First posted 24 Sep 2026cs.AI cs.CLcited by 0

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Running a language model on edge hardware provides private and low-latency reasoning without a network connection, and yet the small models that fit on such devices are unreliable on the tasks computers are expected to handle well, such as arithmetic, algebra, and formal logic problems. We argue that much of this unreliability is avoidable. Many queries appearing to demand reasoning are in fact structurally deterministic and permit fast and exact symbolic solutions. Therefore, forcing a probabilistic model to approximate them sacrifices accuracy and energy for little benefit. We present a neurosymbolic router that classifies each incoming query and dispatches it to the cheapest correct solver, sending structured tasks to deterministic engines and reserving the small language model (SLM) for open-ended word problems. Instead of hand-coding the routing logic, we learn a deterministic finite automaton (DFA) with the L* grammatical inference algorithm, using the SLM as a membership oracle and labeled data as an equivalence oracle. On a Raspberry Pi 4B (8 GB RAM, no GPU), evaluated on 100 untested prompts from DeepMind Mathematics, GSM8K, and RuleTaker, learned routing attains 100% routing accuracy and 98.3% overall accuracy with a 512-token reasoning budget (93.3% on word problems), compared with 72.0% for the strongest agent baseline, Program-of-Thought, and 58.7% for a tool-calling agent given the same solvers. Since formatted queries never reach the model, the router answers them in 1-11 ms and, in its 30-token configuration, runs 8.8x faster and 2.8x more energy-efficient than Program-of-Thought.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 21 of its 31 references that are in the map. First posted on arXiv 24 Sep 2026.

Builds on

21 of 31 references.

References · 21

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  6. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  7. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  8. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  9. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  10. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,076
  11. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  12. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 853
9 more
  1. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 800
  2. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 682
  3. 2022PAL: Program-aided Language Modelscited by 652
  4. 2024Executable Code Actions Elicit Better LLM Agentscited by 564
  5. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 560
  6. 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 521
  7. 2019Analysing Mathematical Reasoning Abilities of Neural Modelscited by 335
  8. 2020Transformers as Soft Reasoners over Languagecited by 270
  9. 2019Neural-Symbolic Computing: An Effective Methodology for Principled Integration of Machine Learning and Reasoningcited by 151

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Resource-Aware Neuro-Symbolic Reasoning for Local Small Language Models
  2. 2025Lobster: A GPU-Accelerated Framework for Neurosymbolic Programming
  3. 2025Hardware-efficient tractable probabilistic inference for TinyML Neurosymbolic AI applications
  4. 2025Current Practices for Building LLM-Powered Reasoning Tools Are Ad Hoc -- and We Can Do Better
  5. 2023LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic Provers
  6. 2023Scallop: A Language for Neurosymbolic Programming
  7. 2024Reliable Reasoning Beyond Natural Language
  8. 2025On Scaling Neurosymbolic Programming through Guided Logical Inference