# Censoring chemical data to mitigate dual use risk

arXiv: https://arxiv.org/abs/2304.10510 · HTML: https://mlsanity.com/paper/2304.10510/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 111 references that are in the map.
- First posted on arXiv 20 Apr 2023.

## Builds on (most cited first, 16 of 16)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [MoleculeNet: A Benchmark for Molecular Machine Learning](https://mlsanity.com/paper/1703.00564/) (2017)
- [Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned](https://mlsanity.com/paper/2209.07858/) (2022)
- [A Watermark for Large Language Models](https://mlsanity.com/paper/2301.10226/) (2023)
- [Federated Machine Learning: Concept and Applications](https://mlsanity.com/paper/1902.04885/) (2019)
- [Responsive Safety in Reinforcement Learning by PID Lagrangian Methods](https://mlsanity.com/paper/2007.03964/) (2020)
- [Low-Shot Learning from Imaginary Data](https://mlsanity.com/paper/1801.05401/) (2018)
- [Emergent autonomous scientific research capabilities of large language models](https://mlsanity.com/paper/2304.05332/) (2023)
- [Low Data Drug Discovery with One-shot Learning](https://mlsanity.com/paper/1611.03199/) (2016)
- [Understanding Double Descent Requires a Fine-Grained Bias-Variance Decomposition](https://mlsanity.com/paper/2011.03321/) (2020)
- [Self-Destructing Models: Increasing the Costs of Harmful Dual Uses of Foundation Models](https://mlsanity.com/paper/2211.14946/) (2022)
- [DGL-LifeSci: An Open-Source Toolkit for Deep Learning on Graphs in Life Science](https://mlsanity.com/paper/2106.14232/) (2021)
- [Avoiding Negative Side Effects due to Incomplete Knowledge of AI Systems](https://mlsanity.com/paper/2008.12146/) (2020)

## Built on by (most cited first, 3 of 3)

- [Are large language models superhuman chemists?](https://mlsanity.com/paper/2404.01475/) (2024)
- [Probing the limitations of multimodal language models for chemistry and materials research](https://mlsanity.com/paper/2411.16955/) (2024)
- [Towards Responsible Governance of Biological Design Tools](https://mlsanity.com/paper/2311.15936/) (2023)

## Abstract (arXiv)

Machine learning models have dual-use potential, potentially serving both beneficial and malicious purposes. The development of open-source models in chemistry has specifically surfaced dual-use concerns around toxicological data and chemical warfare agents. We discuss a chain risk framework identifying three misuse pathways and corresponding mitigation strategies: inference-level, model-level, and data-level. At the data level, we introduce a model-agnostic noising method to increase prediction error in specific desired regions (sensitive regions). Our results show that selective noise induces variance and attenuation bias, whereas simply omitting sensitive data fails to prevent extrapolation. These findings hold for both molecular feature multilayer perceptrons and graph neural networks. Thus, noising molecular structures can enable open sharing of potential dual-use molecular data.