# Semantic Objective Functions: A distribution-aware method for adding logical constraints in deep learning

arXiv: https://arxiv.org/abs/2405.15789 · HTML: https://mlsanity.com/paper/2405.15789/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in General knowledge distillation methods, one of 12 directions in Knowledge distillation.
- Builds on 12 of its 32 references that are in the map.
- First posted on arXiv 3 May 2024.

## Placed in

- Knowledge distillation / [General knowledge distillation methods](https://mlsanity.com/distill/general-knowledge-distillation-methods/)

## Builds on (most cited first, 12 of 12)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Neural Probabilistic Logic Programming in DeepProbLog](https://mlsanity.com/paper/1907.08194/) (2019)
- [A Semantic Loss Function for Deep Learning with Symbolic Knowledge](https://mlsanity.com/paper/1711.11157/) (2017)
- [Harnessing Deep Neural Networks with Logic Rules](https://mlsanity.com/paper/1603.06318/) (2016)
- [Embedding Symbolic Knowledge into Deep Networks](https://mlsanity.com/paper/1909.01161/) (2019)
- [MultiplexNet: Towards Fully Satisfied Logical Constraints in Neural Networks](https://mlsanity.com/paper/2111.01564/) (2021)
- [Deep Learning with Logical Constraints](https://mlsanity.com/paper/2205.00523/) (2022)
- [Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead](https://mlsanity.com/paper/1811.10154/) (2018)
- [Symbolic Logic meets Machine Learning: A Brief Survey in Infinite Domains](https://mlsanity.com/paper/2006.08480/) (2020)
- [Parallel Neurosymbolic Integration with Concordia](https://mlsanity.com/paper/2306.00480/) (2023)

## Built on by (most cited first, 4 of 4)

- [Unlocking the Potential of Generative AI through Neuro-Symbolic Architectures: Benefits and Limitations](https://mlsanity.com/paper/2502.11269/) (2025)
- [Logic Tensor Network-Enhanced Generative Adversarial Network](https://mlsanity.com/paper/2601.03839/) (2026)
- [Neurosymbolic Reasoning Shortcuts under the Independence Assumption](https://mlsanity.com/paper/2507.11357/) (2025)
- [Generate in the Chart, Not on the Boundary: Function-Symbol Grounding for Hard Constraints in LTN-GANs](https://mlsanity.com/paper/2608.21605/) (2026)

## Abstract (arXiv)

Issues of safety, explainability, and efficiency are of increasing concern in learning systems deployed with hard and soft constraints. Symbolic Constrained Learning and Knowledge Distillation techniques have shown promising results in this area, by embedding and extracting knowledge, as well as providing logical constraints during neural network training. Although many frameworks exist to date, through an integration of logic and information geometry, we provide a construction and theoretical framework for these tasks that generalize many approaches. We propose a loss-based method that embeds knowledge-enforces logical constraints-into a machine learning model that outputs probability distributions. This is done by constructing a distribution from the external knowledge/logic formula and constructing a loss function as a linear combination of the original loss function with the Fisher-Rao distance or Kullback-Leibler divergence to the constraint distribution. This construction includes logical constraints in the form of propositional formulas (Boolean variables), formulas of a first-order language with finite variables over a model with compact domain (categorical and continuous variables), and in general, likely applicable to any statistical model that was pretrained with semantic information. We evaluate our method on a variety of learning tasks, including classification tasks with logic constraints, transferring knowledge from logic formulas, and knowledge distillation from general distributions.