Techniques and metrics for evasion attack mitigation

Evasion attacks pose a substantial risk to the application of Machine Learning (ML) in Cybersecurity, potentially leading to safety hazards or security breaches in large-scale deployments. Adversaries can employ evasion attacks as an initial tactic to deceive malware or network scanners using ML, thereby orchestrating traditional cyber attacks to disrupt systems availability or compromise integrity. Adversarial data designed to fool AI systems for cybersecurity can be engineered by strategically selecting, modifying, or creating test instances. This paper presents novel defender-centric techniques and metrics for mitigating evasion attacks by leveraging adversarial knowledge, exploring potential exploitation methods, and enhancing alarm detection capabilities. We first introduce two new evasion resistance metrics: adversarial failure rate (a f r) and adversarial failure curves (a f c). These metrics generalize previous approaches, as they can be applied to threshold classifiers, facilitating analyses for adversarial attacks comparable to those performed with Receiver Operating Characteristics (ROC) curve. Subsequently, we propose two novel evasion resistance techniques (trainset size pinning and model matrix), extending research in keyed intrusion detection and randomization. We explore the application of proposed techniques and metrics to an intrusion detection system as a pilot study using two public datasets, ‘BETH 2021’ and ‘Kyoto 2015’, which are well-established cybersecurity datasets for uncertainty and robustness benchmarking. The experimental results demonstrate that the combination of the proposed randomization techniques consistently produces remarkable improvement over other known randomization techniques.