# Ethic-BERT: An Enhanced Deep Learning Model for Ethical and Non-Ethical Content Classification

arXiv: https://arxiv.org/abs/2510.12850 · HTML: https://mlsanity.com/paper/2510.12850/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 42 references that are in the map.
- First posted on arXiv 14 Oct 2025.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [On the Convergence of Adam and Beyond](https://mlsanity.com/paper/1904.09237/) (2019)
- [A Primer in BERTology: What we know about how BERT works](https://mlsanity.com/paper/2002.12327/) (2020)
- [Aligning AI With Shared Human Values](https://mlsanity.com/paper/2008.02275/) (2020)
- [Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation](https://mlsanity.com/paper/2010.16061/) (2020)
- [Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Do](https://mlsanity.com/paper/2103.11790/) (2021)
- [Dropout Training as Adaptive Regularization](https://mlsanity.com/paper/1307.1493/) (2013)
- [Detection of Hate Speech using BERT and Hate Speech Word Embedding with Deep Model](https://mlsanity.com/paper/2111.01515/) (2021)
- [Spam-T5: Benchmarking Large Language Models for Few-Shot Email Spam Detection](https://mlsanity.com/paper/2304.01238/) (2023)
- [A Chat About Boring Problems: Studying GPT-based text normalization](https://mlsanity.com/paper/2309.13426/) (2023)
- [Macro Ethics Principles for Responsible AI Systems: Taxonomy and Future Directions](https://mlsanity.com/paper/2208.12616/) (2022)
- [Towards Ethics by Design in Online Abusive Content Detection](https://mlsanity.com/paper/2010.14952/) (2020)
- [A Modified AUC for Training Convolutional Neural Networks: Taking Confidence into Account](https://mlsanity.com/paper/2006.04836/) (2020)
- [Hate Speech and Offensive Language Detection using an Emotion-aware Shared Encoder](https://mlsanity.com/paper/2302.08777/) (2023)

## Abstract (arXiv)

Developing AI systems capable of nuanced ethical reasoning is critical as they increasingly influence human decisions, yet existing models often rely on superficial correlations rather than principled moral understanding. This paper introduces Ethic-BERT, a BERT-based model for ethical content classification across four domains: Commonsense, Justice, Virtue, and Deontology. Leveraging the ETHICS dataset, our approach integrates robust preprocessing to address vocabulary sparsity and contextual ambiguities, alongside advanced fine-tuning strategies like full model unfreezing, gradient accumulation, and adaptive learning rate scheduling. To evaluate robustness, we employ an adversarially filtered "Hard Test" split, isolating complex ethical dilemmas. Experimental results demonstrate Ethic-BERT's superiority over baseline models, achieving 82.32% average accuracy on the standard test, with notable improvements in Justice and Virtue. In addition, the proposed Ethic-BERT attains 15.28% average accuracy improvement in the HardTest. These findings contribute to performance improvement and reliable decision-making using bias-aware preprocessing and proposed enhanced AI model.