# Low latency optical-based mode tracking with machine learning deployed on FPGAs on a tokamak

arXiv: https://arxiv.org/abs/2312.00128 · HTML: https://mlsanity.com/paper/2312.00128/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 35 references that are in the map.
- First posted on arXiv 30 Nov 2023.

## Builds on (most cited first, 3 of 3)

- [HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precision](https://mlsanity.com/paper/1905.03696/) (2019)
- [Fast convolutional neural networks on FPGAs with hls4ml](https://mlsanity.com/paper/2101.05108/) (2021)
- [Applications and Techniques for Fast Machine Learning in Science](https://mlsanity.com/paper/2110.13041/) (2021)

## Built on by (most cited first, 7 of 7)

- [hls4ml: A Flexible, Open-Source Platform for Deep Learning Acceleration on Reconfigurable Hardware](https://mlsanity.com/paper/2512.01463/) (2025)
- [AI Benchmark Democratization and Carpentry](https://mlsanity.com/paper/2512.11588/) (2025)
- [Architectural Implications of Neural Network Inference for High Data-Rate, Low-Latency Scientific Applications](https://mlsanity.com/paper/2403.08980/) (2024)
- [Real-Time Cell Sorting with Scalable In Situ FPGA-Accelerated Deep Learning](https://mlsanity.com/paper/2503.12622/) (2025)
- [Scalable FPGA Framework for Real-Time Denoising in High-Throughput Imaging: A DRAM-Optimized Pipeline using High-Level Synthesis](https://mlsanity.com/paper/2508.14917/) (2025)
- [Ultrafast On-Chip Online Learning via Spline Locality in Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2602.02056/) (2026)
- [Design Rules for Extreme-Edge Scientific Computing on AI Engines](https://mlsanity.com/paper/2604.19106/) (2026)

## Abstract (arXiv)

Active feedback control in magnetic confinement fusion devices is desirable to mitigate plasma instabilities and enable robust operation. Optical high-speed cameras provide a powerful, non-invasive diagnostic and can be suitable for these applications. In this study, we process fast camera data, at rates exceeding 100kfps, on textitin situ Field Programmable Gate Array (FPGA) hardware to track magnetohydrodynamic (MHD) mode evolution and generate control signals in real-time. Our system utilizes a convolutional neural network (CNN) model which predicts the n=1 MHD mode amplitude and phase using camera images with better accuracy than other tested non-deep-learning-based methods. By implementing this model directly within the standard FPGA readout hardware of the high-speed camera diagnostic, our mode tracking system achieves a total trigger-to-output latency of 17.6μs and a throughput of up to 120kfps. This study at the High Beta Tokamak-Extended Pulse (HBT-EP) experiment demonstrates an FPGA-based high-speed camera data acquisition and processing system, enabling application in real-time machine-learning-based tokamak diagnostic and control as well as potential applications in other scientific domains.