Ontology-Amplified Distillation and Contextuality Auditing for Sovereign Enterprise Language Models: A Combined Proof-of-Mechanism and Negative-Results Method Study
On the map: Knowledge distillation → On-policy and self-distillation for LLMs
Abstract (arXiv)
Regulated financial institutions operating under data-residency rules need tenant-owned language models that can run inside the institution's perimeter. This paper combines two related FAOS studies into one mechanism-and-control article. First, it reports a reduced-power proof-of-mechanism study of ontology-amplified distillation: a Qwen3.6-27B student is adapted to the Foundation AgenticOS ontology through supervised fine-tuning on frontier-teacher trajectories and ontology-grounded direct preference optimization (DPO), trained locally on a single Apple M5 Max from 47 synthetic, English-language, cross-domain preference pairs. On 40 held-out Vietnamese financial-domain tasks, the distilled student grounds 36 of 40 tasks (grounded rate 0.90; mean ontology term-coverage r_onto = 0.95 on a metric floored at 0.50), equal to the GPT-5 frontier baseline, which also grounds 36 of 40. The outcome is underpowered to establish equivalence: the paired-difference 95% confidence interval spans +/-4 tasks, and the run does not test or show the pre-registered amplification prediction that the student should exceed the frontier. Second, the paper consolidates a contextuality-audit method for enterprise-agent routing. In a separate negative-results pilot, the corrected canonical Contextuality-by-Default degree is zero for all Phase 1.3 groups in both the local-Qwen run and an explicitly labeled Gemma replication check; the useful signal is direct influence and construct coupling, not surviving residual contextuality. Together, the studies pair an ontology-grounded model-building mechanism with a governance diagnostic for deciding when apparent disagreement should trigger prompt standardization, multi-agent synthesis, or human review. The evidence supports neither deployability, safety, superiority, statistical equivalence, nor a contextuality-positive routing rule.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in On-policy and self-distillation for LLMs, one of 12 directions in Knowledge distillation. Builds on 15 of its 29 references that are in the map. First posted on arXiv 11 Jul 2026.
Builds on
15 of 29 references.
References · 15
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
- 2017Deep reinforcement learning from human preferencescited by 3,912
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
- 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
- 2024Large Language Model based Multi-Agents: A Survey of Progress and Challengescited by 769
- 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 668
- 2023Zephyr: Direct Distillation of LM Alignmentcited by 506
- 2020Neurosymbolic AI: The 3rd Wavecited by 255
- 2023LLMs4OL: Large Language Models for Ontology Learningcited by 77
- 2024OG-RAG: Ontology-Grounded Retrieval-Augmented Generation For Large Language Modelscited by 23
3 more
- 2025DistilQwen2.5: Industrial Practices of Training Distilled Open Lightweight Language Modelscited by 10
- 2025OntoTune: Ontology-Driven Self-training for Aligning Large Language Modelscited by 8
- 2026Ontology-Constrained Neural Reasoning in Enterprise Agentic Systems: A Neurosymbolic Architecture for Domain-Grounded AI Agentscited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Auditable by Construction: An Ontology-Driven Framework for Trustworthy LLM Analytics in Enterprise Finance
- 2026FinReporting: An Agentic Workflow for Localized Reporting of Cross-Jurisdiction Financial Disclosures
- 2026Toward Pre-Deployment Assurance for Enterprise AI Agents: Ontology-Grounded Simulation and Trust Certification
- 2026FinReasoning: A Hierarchical Benchmark for Reliable Financial Research Reporting
- 2023FinEval: A Chinese Financial Domain Knowledge Evaluation Benchmark for Large Language Models
- 2026Corporate Language Model (CLM): Transforming Tacit and Fragmented Enterprise Knowledge into a Sovereign, Auditable, and Executable Corporate Intelligence Layer
- 2026Mint-Agent: Introducing Finance-Native Agentic Foundation Models
- 2025Agentar-Fin-R1: Enhancing Financial Intelligence through Domain Expertise, Training Efficiency, and Advanced Reasoning