A Study of 118 Field Machines Finds Isolation Forest Collapsing to 0.12 F1

A preprint submitted on 15 April 2026 by researchers at Universität Augsburg and Technische Hochschule Augsburg compares unsupervised anomaly detectors on a proprietary dataset of multivariate time series from 118 field-deployed machines of the same type, covering 334 individual process instances. Only newly commissioned machines were used and recordings were limited to the first two months of operation, so the training data is free of already-degraded behaviour.

Isolation Forest — one of the most commonly reached-for detectors in industrial monitoring, and a default in several commercial platforms — averaged an F1 of 0.120 with a standard deviation of 0.126. Five runs detected no anomalies at all. The best single run reached 0.308, catching 4 of 22 anomalous processes; precision was 1.0 in the runs that fired, but maximum recall was 0.182. In other words it was right when it spoke and it almost never spoke.

Autoencoders performed very differently on the same data. A temporal convolutional autoencoder reached 0.991 ± 0.009, a GRU autoencoder 0.918 ± 0.066 and an LSTM autoencoder 0.853 ± 0.102, with variational variants between 0.876 and 0.968. The authors' conclusion is about architecture rather than capacity: convolutional models beat recurrent ones because real manufacturing processes are non-periodic with variable phase ordering, unlike the NASA Turbofan and bearing test-rig benchmarks that most published results are tuned against.

The caveats belong in any summary. The held-out labelled evaluation set is small — 46 complete process instances, 22 anomalous and 24 normal — and detection thresholds were optimised after the fact across precision, recall, accuracy and F1, so the authors present these as best achievable rather than as deployment thresholds. Six autoencoder variants were each trained 50 times with different seeds and hyperparameters.

The transferable lesson is the gap between benchmark and plant. A detector that performs well on a periodic test rig can fail on a shop floor where normal is a moving target, and the only way to know which you have is to evaluate on your own machines.