Robustness of Anomaly Detection Models for Industrial Control Systems under Training-Time Data Contamination

Anomaly-based detection has become a major line of research in ICS because process behavior is highly structured, while comprehensive attack labels are often limited or costly to obtain. The SWaT testbed and dataset have become widely used benchmarks because they provide a realistic and reproducible platform for evaluating process-aware attack-detection methods [15, 16]. Prior SWaT studies include recurrent models trained on normal data [1], convolutional models [6], unsupervised and one-class approaches [7], sequence-to-sequence architectures [8], lightweight neural and PCA-based detection [9], and methodology-oriented work emphasizing preprocessing, thresholding, and evaluation [17].

Together, these studies support two recurring conclusions. First, anomaly-based monitoring is a natural fit for ICS settings because normal process behavior is repetitive, correlated, and usually easier to collect than diverse attack traces. Second, there is no universally dominant detector family. Broader reviews of machine learning for ICS security reach similar conclusions while identifying persistent challenges in cross-paper comparability, dataset dependence, and deployment realism [2, 3].

Our study also intersects with benchmark design for anomaly and time-series anomaly detection. Prior work shows that detector rankings can shift substantially with metric choice, hyperparameter treatment, and evaluation protocol [18, 19, 20], while time-series studies highlight temporal leakage, split construction, and inconsistent evaluation rules [21, 22]. These observations reinforce the need to interpret detector performance through both the model family and the evaluation pipeline used to generate the reported results.¹¹ 1 Code and experiment configurations for this paper are publicly available at: https://github.com/ANTS-OntarioTechU/Robustness-Anomaly-Detection-ICS-Data-Contamination

Although adversarial machine learning has received increasing attention in ICS security, most prior work has focused on evasion rather than poisoning. Zizzo et al. studied adversarial attacks against time-series intrusion-detection models for ICS and showed that recurrent detectors can be manipulated even when an attacker controls only a subset of variables [10]. Erba et al. showed that realistic ICS manipulation must satisfy process and physical constraints, making the attack space substantially different from conventional adversarial examples [11]. Anthi et al. likewise demonstrated that adversarial perturbations can materially reduce machine-learning-based cyber defenses in industrial environments [12]. Collectively, these studies show that strong clean-data detection performance should not be interpreted as evidence of adversarial robustness.

Compared with evasion, training-time corruption of ICS anomaly detectors has received much less direct study. Kravchik, Biggio, and Shabtai analyzed poisoning attacks against online-trained neural anomaly detectors for ICS [13]. This work was later extended to a practical evaluation of poisoning attacks on online anomaly detectors, again emphasizing online adaptation as the primary threat surface [14]. That setting is highly relevant, but differs from the offline training regime used in many operational pipelines, where models are built or periodically refreshed from curated historian logs, archived process traces, and exported datasets before deployment. Online poisoning studies therefore do not fully characterize contamination of the normal training pool during offline model development.

Outside ICS, the broader adversarial machine-learning literature has shown that training-time corruption can substantially degrade anomaly-detection and classification systems. Rubinstein et al. demonstrated that poisoning can distort anomaly detectors, particularly PCA-based detectors, by shifting the learned model of normality [4]. Other work developed poisoning attacks against SVMs and deep learning, certified defenses, targeted clean-label attacks, and broader surveys [23, 24, 25, 26, 5]. Recent CPS studies also address provable mitigation and dynamic backdoors [27, 28]. These studies provide the conceptual basis for training-time vulnerability, but primarily address optimization-based poisoning rather than the contamination-style attack families considered here.

Prior ICS studies provide strong evidence that anomaly-based detectors can perform effectively under clean conditions [1, 6, 7, 8, 9, 17], while adversarial ICS studies have focused mainly on evasion or poisoning tied to online retraining [10, 11, 12, 13, 14]. What remains insufficiently characterized is the comparative robustness of a broad set of anomaly-based ICS detectors under offline training-time contamination within a unified benchmarking framework.

More specifically, to the best of our knowledge, existing literature does not provide a consistent comparison across heterogeneous anomaly-detector families under the same contamination definitions, budgets, threshold-calibration procedure, and clean held-out evaluation protocol. It also remains unclear whether detectors that appear strong on clean benchmark data retain that advantage once the normal training pool is contaminated. The novelty of this study is a unified, configuration-level robustness benchmark across heterogeneous anomaly-detector families under common training pools, contamination budgets, clean calibration data, held-out testing, and evaluation rules.

We consider an offline anomaly-detection pipeline for ICS under a normal-only training regime. Each detector is trained to model nominal process behavior from a clean training pool, while model selection and threshold calibration use a labeled validation split and final performance is measured on a held-out test split that is never modified during contamination. This setup reflects a practical workflow in which historical process data are curated, preprocessed, and used to develop anomaly detectors before deployment.

The adversary’s objective is to degrade the detector’s ability to distinguish attacks from normal behavior at test time while preserving the appearance of a plausible training workflow. More specifically, the attacker seeks to increase false negatives, reduce F1-score and recall, and distort the learned notion of normality so that malicious behavior is more likely to be accepted as benign.

We assume a gray-box adversary with training-time access only. The attacker knows the feature representation and general anomaly-detection pipeline, but does not modify the clean validation or test sets and does not interact with the deployed detector at inference time. Depending on the contamination strategy, the adversary either injects attack samples into the nominal training pool or perturbs selected normal training observations. The similarity-targeted strategy assumes more informed sample selection than random contamination, while all evaluated strategies remain weaker than white-box optimization-based attacks. The budget is at most 10% of the clean training-pool size.

For clarity, “uncontaminated” means that no training-time attack is applied to validation or test observations; it does not mean that the labeled validation split is attack-free. The restrictions to clean calibration and test data isolate training-time contamination effects, but may be optimistic relative to workflows in which curation errors or historian compromise propagate beyond the training data.

Offline ICS datasets can absorb attack traces or distorted values through annotation, alignment, curation, export, or preprocessing failures. These failures need not alter the running process; they affect a normal-only detector through data presented as benign. We retain poisoning for continuity with the literature, but evaluate contamination-style rather than gradient-driven worst-case attacks. Figure 1 summarizes the workflow.

Two practical corruption paths motivate the evaluated strategies. First, attack-period observations may be included in a nominal archive through missing labels, event-window misalignment, or unsafe merging of exports. Second, stored normal observations may be modified through historian compromise or preprocessing faults without changing the number of records. Injection attacks model the first path, while feature-noise injection models the second. The benchmark does not claim that these mechanisms exhaust real data-governance failures; they provide repeatable stress tests with a common budget.

Let the clean normal training pool be $D_{\mathrm{clean}}=\{x_{i}\}_{i=1}^{N}$ and the available attack pool be $A=\{a_{j}\}_{j=1}^{M}$. For a training-pool-relative budget $p$, the adversary manipulates $k=\lfloor pN\rfloor$ samples. For the injection attacks, appending $k$ samples yields a final injected fraction $k/(N+k)$ (approximately 9.09% when $p=10\%$); feature noise instead modifies $k$ of the original $N$ samples. Thus, $p$ denotes a common budget relative to the clean pool rather than an identical final-set fraction across attack families.

For the pointwise detectors and AE, the attack pool is the attack-labeled portion of the stratified training fold and is therefore disjoint from their validation and test observations. LSTM-AE uses this pointwise training-fold attack pool to contaminate its separate contiguous normal training block, as detailed in Section IV-A. This controlled resource isolates training contamination within each implemented protocol but is optimistic relative to end-to-end data-governance failures.

The injected pool should therefore be understood as an evaluation resource rather than an assumption that an operational attacker possesses a perfectly curated attack corpus. It enables controlled comparisons across budgets and seeds. Similarly, the clean validation and test assumption prevents threshold corruption from being conflated with model corruption, but may underestimate failures in pipelines where the same data-quality problem propagates across all splits.

SWaT contains multivariate time-series data from normal operation and attack scenarios [15, 16]. Timestamp columns are removed, source files are aligned on common features, values are coerced to numeric form, invalid rows are dropped, and seven constant columns (P202, P301, P401, P404, P502, P601, and P603) are removed. After the implemented preprocessing, the pointwise benchmark pool contains 449,919 samples, 44 features, and a 12.14% attack ratio. MinMax parameters are fitted on training data only and then applied to validation and test data.

Pointwise detectors and AE use a stratified 70/15/15 split of the pooled normal- and attack-file observations. Only normal samples from the training fold are used for detector fitting; labeled validation observations calibrate the F1-oriented threshold. LSTM-AE instead preserves the source-file order: the first 70% of the normal-operation file is used for training, the next 15% for early stopping, and the final 15% supplies normal test observations. The first half of the attack-period file is used for threshold selection and the second half for testing. For contaminated LSTM-AE runs, attack observations are drawn from the attack-labeled portion of the pointwise training fold and appended to the contiguous normal training block. Windows of length $W=30$ are then constructed independently within the training, early-stopping, threshold-selection, and test arrays, so no window crosses a boundary between these arrays. A window is labeled anomalous if any included time step is attacked. This temporal protocol is stricter, but its sequence-level labels are not identical to the pointwise evaluation and must be considered in cross-family comparisons.

Preprocessing is fitted without using validation or test statistics. Timestamp columns are removed, the normal and attack files are aligned on their common process variables, and nonnumeric or invalid rows are discarded before splitting. The seven removed process variables are constant under the available traces and therefore provide no discriminatory variation. For the pointwise protocol, stratification preserves the attack ratio in validation and test data; only the normal portion of the training fold is passed to a detector. For LSTM-AE, preserving order avoids constructing windows from unrelated time points and prevents a window from crossing the boundary between the normal training sequence and the attack period.

The two protocols consequently use different units of evaluation. A pointwise model assigns one score to one 44-dimensional observation. LSTM-AE assigns reconstruction errors to overlapping windows and labels a window anomalous when at least one included time step belongs to an attack interval. We report this distinction because it affects both the number of evaluated instances and the meaning of a false negative. Cross-family comparisons remain useful at the implemented-configuration level, but should not be read as if every detector received an identical sample representation.

The retained models are Isolation Forest (IForest), One-Class SVM, Local Outlier Factor (LOF), Cluster-Based Local Outlier Factor (CBLOF), KNN, Histogram-Based Outlier Score (HBOS), PCA, Minimum Covariance Determinant (MCD), Angle-Based Outlier Detection (ABOD), a feedforward autoencoder (AE), and LSTM-AE [29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39]. Classical detectors use PyOD; neural detectors use PyTorch. PCA and SVM are tuned, while other classical models use default or near-default configurations. AE and LSTM-AE use staged hyperparameter searches with multi-seed confirmation.

Table III summarizes the heterogeneous scoring mechanisms, final retained configurations, and training inputs, allowing contamination sensitivity to be compared across incompatible definitions of normality without duplicating training-scale information.

The classical benchmark uses PyOD implementations with a common normal-only fitting workflow. IForest, HBOS, CBLOF, and PCA use the full pointwise normal training pool, while SVM, LOF, KNN, ABOD, and MCD use a fixed 50,000-point uniform subsample to control super-linear memory or distance-computation costs. IForest uses 100 trees and the default 256-sample tree subsample. LOF uses 20 neighbors, KNN scores distance to the fifth neighbor, CBLOF uses eight clusters, and ABOD uses its fast 10-neighbor configuration. PCA retains 90% variance, while SVM uses an RBF kernel with $\nu=0.01$ and gamma = scale.

Table III records the retained settings needed to interpret and reproduce each reported configuration. The PyOD contamination argument is fixed at 0.05 during construction where applicable, but it does not define the final operating threshold. Every detector is converted to binary decisions using the separate validation-calibration procedure described below. Thus, the benchmark’s 1–10% attack budgets describe training-pool manipulation and should not be confused with the library’s internal contamination parameter.

Tuning follows a sensitivity-first policy. A classical parameter family is promoted only when a preliminary change improves F1 by at least 0.03 or reduces FNR by at least 0.05 relative to its default slice. Only PCA and SVM cross this promotion threshold; other classical detectors retain default or near-default configurations. This conservative policy avoids tailoring every model extensively to one benchmark, but it also means that the reported ordering is configuration-level evidence rather than a definitive comparison of optimally tuned families.

The neural models receive staged searches because architecture and optimization choices materially affect their performance. AE is evaluated over 426 configurations spanning architecture, optimizer, regularization, threshold, and scaling choices. LSTM-AE is evaluated over 216 configurations spanning sequence length, hidden capacity, and training settings. Final candidates are re-ranked over three seeds using a validation-only composite of clean F1-score and F1-score under 10% similarity-targeted injection. All hyperparameter screening, staged selection, and multi-seed re-ranking use validation performance only; the held-out test set is accessed only after selecting the final configuration. Table IV records the resulting policy and configurations.

Both neural models minimize mean-squared reconstruction error, use gradient clipping at 1.0, and apply early stopping with patience 15. AE permits up to 100 epochs and LSTM-AE up to 30. The LSTM-AE search also exposes an interaction that a one-factor sweep would miss: $W=30$ and hidden size 256 are not the strongest isolated changes, but their joint configuration ranks highest after staged multi-seed evaluation. Dropout values are not interpreted for the selected single-layer LSTM because PyTorch applies recurrent dropout only between stacked recurrent layers.

Experiments were run on the Narval cluster of the Digital Research Alliance of Canada. Classical jobs used six Intel Xeon CPU cores and 48 GiB of RAM; neural jobs additionally used one NVIDIA A100-SXM4-40 GB GPU. Fixed seeds 42, 123, and 456 are applied to splitting, contamination, initialization, and training. Each run is checkpointed independently before aggregation, supporting restartable execution and consistent generation of the reported tables and figures.

All detectors output continuous anomaly scores. An F1-oriented threshold is selected on validation data and applied unchanged to held-out test data. The primary metric is F1-score; FNR is reported because missed attacks are safety-relevant. Experiments use seeds 42, 123, and 456 and training-pool-relative budgets of 1%, 3%, 5%, and 10%. Execution produced 430 successful runs: 33 retained clean baselines, 396 retained contamination runs, and one SOD clean baseline used only to document exclusion. SOD was removed from the main benchmark because its clean ROC-AUC indicated inverted score ordering under the evaluated configuration, making contamination-degradation analysis ill-posed [40]. Its available baseline also showed severe precision collapse (F1 $=0.2013$, precision $=0.1130$, recall $=0.9218$) and a 2011.4-s training time; no SOD contamination result is included in the main analysis. All reported comparative analyses therefore use 429 retained runs.

For every clean or contaminated model within each seed and detector configuration, a new threshold is calibrated using only the same clean labeled validation split. That threshold is then frozen for the corresponding held-out test evaluation. This prevents test-set optimization and measures whether each contaminated model’s anomaly-score distribution remains compatible with an operating point selected without contaminated calibration data. Precision, recall, F1-score, FNR, ROC-AUC, and average precision are retained in the experiment outputs; F1 and FNR are emphasized because the former balances the two error directions while the latter directly exposes missed attacks.

Table V combines clean performance, measured training cost, and worst-case F1 loss. LSTM-AE has the highest clean F1-score ($0.8813$), followed by LOF, ABOD, and AE. Yet the clean-data ranking changes substantially under contamination. LOF, KNN, and ABOD each lose more than 0.64 F1 in their worst condition, while IForest, HBOS, PCA, and SVM remain within 0.068 of their clean baselines. AE and LSTM-AE occupy an intermediate position.

The clean scores are descriptive means over three seeds rather than significance-tested orderings. The leading group is narrow: LSTM-AE, LOF, ABOD, and AE all lie between 0.867 and 0.881. PCA and SVM are mid-table at approximately 0.813–0.814, but require far less training time than ABOD, KNN, AE, or LSTM-AE. Clean evaluation therefore defines the starting point for robustness analysis but does not identify a single operational winner.

The standard deviations also distinguish stable clean estimates from seed-sensitive ones. Most detectors vary by less than 0.01 F1 across the three seeds, whereas LOF and MCD vary more noticeably because their fitting pools are subsampled. These differences are small relative to the largest contamination losses, but they reinforce why degradation is computed from seed-averaged contaminated and clean scores rather than from a single favorable run.

Figure 2 shows distinct attack responses. Feature noise at $\sigma=0.15$ changes little, whereas 1% random injection reduces LOF, KNN, ABOD, and LSTM-AE from 0.876, 0.848, 0.874, and 0.881 to approximately 0.398, 0.335, 0.305, and 0.440. AE degrades progressively; similarity-targeted injection becomes severe for selected distance- and density-based detectors at 10%.

The earliest failures are operationally important because a small clean-pool-relative budget is sufficient to reverse the clean ranking. LOF, KNN, and ABOD begin among the five strongest clean configurations, yet at 1% random injection each loses more than half of its original F1-score. Their reliance on local neighborhoods, distances, or angles makes them sensitive when attack samples are incorporated into the reference geometry. In contrast, the more aggregated representations of PCA, HBOS, IForest, and SVM change much less under the same evaluated contamination.

The neural response is neither uniformly fragile nor uniformly robust. AE deteriorates gradually and reaches its worst condition at 10% random injection, where F1 falls to approximately 0.566. LSTM-AE has the highest clean F1-score but drops to approximately 0.440 at only 1% random injection. Its response to similarity-targeted injection is much milder through 5%, indicating that the centroid-based pointwise selection rule does not transfer directly into a worst-case sequence attack. The neural results therefore support an intermediate robustness interpretation and demonstrate why attack-family labels should not be treated as equivalent across data representations.

Figure 3 complements the F1 analysis with missed-detection behavior and is discussed in Section V-C. In Figs. 2–4, the labels HISTOGRAM, CLUSTER, and AUTOENCODER denote HBOS, CBLOF, and AE, respectively; axes labeled “contamination rate” report the clean-pool-relative budget $p$ defined in Section III-B.

The heatmap in Fig. 4 makes the model dependence explicit. IForest, HBOS, PCA, and SVM remain close to their clean values. Random injection sharply harms LOF, KNN, ABOD, and LSTM-AE across budgets, whereas AE degrades more gradually. Similarity-targeted injection produces delayed failures for LOF, KNN, and ABOD. These results show why a clean-only leaderboard is inadequate for adversarial model selection.

The three strategies produce qualitatively different degradation patterns. Random injection is the dominant family overall: it causes the earliest large drops and affects both pointwise neighborhood methods and the sequence model. Because the injected samples span the available attack pool rather than only its center-nearest region, they introduce diverse malicious states into the normal reference data. This pattern is consistent with an expansion or distortion of the learned nominal region for detectors whose scores depend strongly on the training geometry.

Similarity-targeted injection is more detector-dependent. At low budgets it is often less harmful than random injection, but LOF, KNN, and ABOD show delayed failures as the budget increases. The centroid rule selects attack samples that are globally close to the average clean state, yet global proximity does not guarantee local proximity in every detector’s representation. The strategy should therefore be viewed as a repeatable stealth-oriented heuristic. It does not establish that random sampling is intrinsically stronger than an optimized targeted attack.

Feature-noise injection remains close to the clean baseline across the grid. Three aspects limit the conclusion: the magnitude is fixed at $\sigma=0.15$, perturbations are clipped to the normalized range, and only selected normal observations are modified. The result shows that the evaluated models tolerate this bounded configuration substantially better than attack-sample injection. It does not show general robustness to biased, coordinated, temporally persistent, or physically constrained sensor corruption.

Figure 3 shows that FNR should be interpreted together with F1-score rather than in isolation. ABOD, KNN, and LOF exceed approximately 0.70 FNR in at least one contamination setting, while LSTM-AE reaches a lower but still concerning peak under random injection. By contrast, PCA, SVM, HBOS, AE, and MCD remain in the lower-FNR group, and IForest stays comparatively stable despite its lower clean baseline.

At some contamination rates, a lower FNR may reflect overly permissive decision behavior, which reduces missed detections at the cost of precision. For that reason, FNR is most informative when read together with F1-score rather than as a standalone safety measure.

Taken together, the results show that the ordering by clean-data F1-score is materially different from the ordering implied by robustness and training cost. LSTM-AE, LOF, ABOD, and AE form the strongest clean-data group. However, several of these models remain fragile once the normal training pool is contaminated, particularly LOF, KNN, and ABOD, and to a lesser but still meaningful degree AE and LSTM-AE. By contrast, PCA and SVM do not lead the clean baseline table, yet both combine moderate clean performance with small worst-case F1 loss and much lower training cost. IForest is also notable for its stability across the contamination grid, although its clean baseline remains lower than that of PCA and SVM.

The most defensible practical conclusion is therefore not that one detector dominates on every metric, but that PCA and SVM provide a favorable observed balance among clean performance, contamination robustness, safety-relevant FNR behavior, and computational cost under the evaluated setup. Because hardware and training scales differ, this balance is configuration-specific rather than a formal multi-objective ranking. AE and LSTM-AE remain intermediate rather than robust, and their contamination-time behavior is less dependable than that of the most stable classical configurations.

Several limitations should be kept in mind when interpreting the benchmark. First, the study is conducted on a single ICS dataset, so the findings should be interpreted at the benchmark level rather than as population-wide claims about all safety-critical ICS environments. Second, the evaluated attack families are contamination-style training-time scenarios rather than optimization-based worst-case poisoning attacks. Third, the benchmark assumes clean validation and test splits, which isolates training-time contamination effects but may be optimistic relative to workflows in which curation errors propagate beyond the training data. Fourth, the benchmark does not apply equally deep hyperparameter optimization to every detector, so cross-detector comparisons should be interpreted as configuration-level robustness evidence rather than definitive family-level rankings. Finally, the pointwise detectors and sequence model use different split structures: the pointwise random split can place temporally adjacent observations in different folds, whereas LSTM-AE preserves a contiguous protocol. The limited feature-noise effect applies only at $\sigma=0.15$.