Explainable AI For Insider Threat Detection: Balancing Transparency With Privacy In Enterprise Systems

Abstract

Insider threat detection has remained an extremely challenging security threat in enterprise systems and applications worldwide. This is mainly due to the complexity of addressing security threats launched by legally entitled users of computer systems. Recent strides in the application of Artificial Intelligence (AI), particularly in the context of enhancing the accuracy of cybersecurity threats in enterprise computer environments and applications, are notable in the literature. However, the main disadvantage of AI-based security detection models and applications is the black box effect associated with the functionality of most of these models, posing potential issues in establishing trust in AI models and applications in the context of establishing the privacy of users of AI-based enterprise applications. This study attempts to establish the application of Explainable Artificial Intelligence (XAI) models in the context of establishing the detection of various security threats in enterprise computer applications. From the perspective of technical implementation models and applications, the proposed AI integration framework focusing on the context of the proposed enterprise activity logs is attempted in the context of real-world-inspired enterprise activity scenarios. To accomplish these tasks, several XAI methods, e.g., SHAP values, rule-based explanations, and feature importance analysis, have been adopted to demonstrate potential means through which security analysts may comprehend and validate the results provided by an insider threat detection system, ensuring their actions do not reveal sensitive personal and organizational information. Overall, the experiment results indicate the possible advantages realized from the adoption of an XAI-based detection system with improved confidence for the analysts and a reduction in the number of false positives in the investigation processes, as well as developing a competitive detection feature and compliance with the data privacy constraint. Additionally, the experiment outcomes demonstrate the following crucial design parameters for the implementation of an XAI-based insider threat detection system to assist security analysts and security developers towards developing the overall design and security contributions with a better balance between accurate detection and data privacy and the avoidance of any possible conflicts between the detection and “explanation” processes.