mlsanityData to 14 Oct 2026

K-OPSD: Verifiable On-Policy Self-Distillation for Post-Training Vision-Language Models on AEC Drawings

arXiv 2609.34082 ↗First posted 28 Sep 2026cs.AI cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Chart understanding and anomaly detection also fits: Embodied and 3D vision-language models

also in 1 more
Abstract (arXiv)

Interpreting architecture, engineering, and construction (AEC) drawings is hard for general Multimodal Large Language Models (MLLMs) and vision-language models (VLMs). We introduce K-OPSD, a VLM post-training methodology for improving AEC drawing understanding. Building on On-Policy Self-Distillation (OPSD) with verifiable supervision, we construct a teacher from the model's own best-of-N generations, certified by a process-level verifier, and rescue failed prompts by resampling under a hint that exposes the verified answer. We then perform an on-policy model update by training on verified completions with a cross-entropy inner-loss, outperforming the bounded token-wise generalized Jensen-Shannon divergence (JSD) used by on-policy distillation. Using K-OPSD, we fine-tune Qwen3-VL models on the AECV-Bench dataset. The resulting models attain the top average judge score (0.819) and combined accuracy (0.738), achieving competitive results against open-source baseline models. The recipe transfers to the out-of-domain ArchCAD dataset, where the 8B model gains most. We present the verifier suite and the continual learning and self-improving pipeline, our results provide preliminary evidence that verifier-guided self-distillation is a promising route toward more reliable machine reading of architecture drawings.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 10 of its 16 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

10 of 16 references.

References · 10

  1. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 939
  2. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 500
  3. 2023Scaling Relationship on Learning Mathematical Reasoning with Large Language Modelscited by 427
  4. 2023Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Modelscited by 349
  5. 2025Large Language Model-Driven Code Compliance Checking in Building Information Modelingcited by 10
  6. 2024Text2BIM: Generating Building Models Using a Large Language Model-based Multi-Agent Frameworkcited by 9
  7. 2024Using Large Language Models for the Interpretation of Building Regulationscited by 6
  8. 2025CEQuest: Benchmarking Large Language Models for Construction Estimationcited by 6
  9. 2024Large Language Models in Fire Engineering: An Examination of Technical Questions Against Domain Knowledgecited by 5
  10. 2026FloorplanVLM: A Vision-Language Model for Floorplan Vectorizationcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026AECV-Bench: Benchmarking Multimodal Models on Architectural and Engineering Drawings Understanding
  2. 2025VOLD: Reasoning Transfer from LLMs to Vision-Language Models via On-Policy Distillation
  3. 2026Seeing Before Reasoning: Decoupling Perception and Reasoning for Shortcut-Resilient Multimodal On-Policy Self-Distillation
  4. 2026On-Policy Distillation for Vision-Language Model Adaptation, an Effective Paradigm on Low-Quality Multimodal Data
  5. 2026Self-Boosting Vision-Language Models with Noisy Student On-Policy Self-Distillation
  6. 2026Benchmarking Deep Learning Approaches for AEC Engineering Drawing Layout Detection and Information Extraction
  7. 2026MMArch: Benchmarking Multimodal Reasoning Grounded in Architectural Evidence
  8. 2026OPD-V: Visual On-Policy Self-Distillation with Modality Balance