# Toward smart composites: small-scale, untethered prediction and control for soft sensor/actuator systems

arXiv: https://arxiv.org/abs/2205.10940 · HTML: https://mlsanity.com/paper/2205.10940/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 58 references that are in the map.
- First posted on arXiv 22 May 2022.

## Builds on (most cited first, 9 of 9)

- [MCUNet: Tiny Deep Learning on IoT Devices](https://mlsanity.com/paper/2007.10319/) (2020)
- [TensorFlow Lite Micro: Embedded Machine Learning on TinyML Systems](https://mlsanity.com/paper/2010.08678/) (2020)
- [CMSIS-NN: Efficient Neural Network Kernels for Arm Cortex-M CPUs](https://mlsanity.com/paper/1801.06601/) (2018)
- [TVM: An Automated End-to-End Optimizing Compiler for Deep Learning](https://mlsanity.com/paper/1802.04799/) (2018)
- [Modeling and Control of Soft Robots Using the Koopman Operator and Model Predictive Control](https://mlsanity.com/paper/1902.02827/) (2019)
- [Distributed Machine Learning in Materials that Couple Sensing, Actuation, Computation and Communication](https://mlsanity.com/paper/1606.03508/) (2016)
- [Materials that make robots smart](https://mlsanity.com/paper/1711.00537/) (2017)
- [Embedded Neural Networks for Robot Autonomy](https://mlsanity.com/paper/1911.03848/) (2019)
- [High-bandwidth nonlinear control for soft actuators with recursive network models](https://mlsanity.com/paper/2101.01139/) (2021)

## Built on by (most cited first, 1 of 1)

- [On-Device Soft Sensors: Real-Time Fluid Flow Estimation from Level Sensor Data](https://mlsanity.com/paper/2311.15036/) (2023)

## Abstract (arXiv)

We present formulation and open-source tools to achieve in-material model predictive control of sensor/actuator systems using learned forward kinematics and on-device computation. Microcontroller units (MCUs) that compute the prediction and control task while colocated with the sensors and actuators enable in-material untethered behaviors. In this approach, small parameter size neural network models learn forward kinematics offline. Our open-source compiler, nn4mc, generates code to offload these predictions onto MCUs. A Newton-Raphson solver then computes the control input in real time. We first benchmark this nonlinear control approach against a PID controller on a mass-spring-damper simulation. We then study experimental results on two experimental rigs with different sensing, actuation and computational hardware: a tendon-based platform with embedded LightLace sensors and a HASEL-based platform with magnetic sensors. Experimental results indicate effective high-bandwidth tracking of reference paths (greater than or equal to 120 Hz) with a small memory footprint (less than or equal to 6.4% of flash memory). The measured path following error does not exceed 2mm in the tendon-based platform. The simulated path following error does not exceed 1mm in the HASEL-based platform. The mean power consumption of this approach in an ARM Cortex-M4f device is 45.4 mW. This control approach is also compatible with Tensorflow Lite models and equivalent on-device code. In-material intelligence enables a new class of composites that infuse autonomy into structures and systems with refined artificial proprioception.