mlsanityData to 14 Oct 2026

Constraint-Anchored Reasoning Traces

arXiv 2607.16727 ↗First posted 18 Jul 2026cs.AI cs.CVcited by 0

On the map: LLM reasoning → Reasoning benchmarks and evaluation

also in 1 more
Abstract (arXiv)

Autoregressive multimodal large language models (MLLMs) suffer from error snowballing: a single incorrect inference early in a chainof-thought (CoT) trace corrupts all downstream reasoning. We find that in state-of-the-art open-source MLLMs, once the first error occurs, the reasoning cascades into failure across all remaining steps in 65% of such cases (a metric we term the snowball rate). Existing mitigations-sampling multiple chains, post-hoc self-verification, or full program synthesis-either lack symbolic grounding, catch errors too late, or sacrifice the flexibility of natural language reasoning. We propose Constraint-Anchored Reasoning Traces (CART), a neuro-symbolic framework that trains MLLMs to interleave natural language reasoning steps with symbolic constraint assertions: lightweight, machine-checkable statements about visual content (e.g., count(red_objects) = 3). A dual-pronged Constraint Propagation Module-combining a learned neural grounding head with Boolean Constraint Propagation-continuously verifies these anchors against extracted visual features and checks their mutual logical consistency. When a contradiction is detected, a backtrack controller halts generation and reverts to the last consistent checkpoint, preventing error propagation. A variable-frequency emission mechanism allows the model to adaptively control anchor density, avoiding trace bloat. We construct 218K training instances by augmenting GQA, CLEVR-CoGenT, and VCR with ground-truth constraint annotations derived from scene graphs, and fine-tune open-source MLLMs (LLaVA-NeXT, Qwen2-VL) via LoRA. On five benchmarks, CART reduces the snowball rate from 0.65 to 0.14, improves GQA accuracy by +4.6 percentage points over trainingonly baselines, and achieves 89.1 F1 on POPE-all with at most 18% inference overhead.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 7 of its 34 references that are in the map. First posted on arXiv 18 Jul 2026.

Builds on

7 of 34 references.

References · 7

  1. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  2. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
  3. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
  4. 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 752
  5. 2023Faithful Chain-of-Thought Reasoningcited by 288
  6. 2024Language Model Cascades: Token-level uncertainty and beyondcited by 121
  7. 2024Seeing is Believing: Mitigating Hallucination in Large Vision-Language Models via CLIP-Guided Decodingcited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Reasoning Large Language Model Errors Arise from Hallucinating Critical Problem Features
  2. 2025Eidoku: A Neuro-Symbolic Verification Gate for LLM Reasoning via Structural Constraint Satisfaction
  3. 2025PCRLLM: Proof-Carrying Reasoning with Large Language Models under Stepwise Logical Constraints
  4. 2024Reliable Reasoning Beyond Natural Language
  5. 2026Imbuing Large Language Models with Bidirectional Logic for Robust Chain Repair
  6. 2025Turning Drift into Constraint: Robust Reasoning Alignment in Non-Stationary Multi-Stream Environments
  7. 2026MMPCBench: Benchmarking Multimodal Large Language Models on Proactive Critique of Flawed Inputs
  8. 2026Pushing the Boundaries of Natural Reasoning: Interleaved Bonus from Formal-Logic Verification