A Scalable HPC Insider Threat Monitoring System

The broader impact/commercial potential of this Small Business Innovation Research (SBIR) Phase I project is to reduced the incidence of insider cyber-enabled crime and insider cyber-related espionage on HPC systems. With the rapid increase in cyber-crime, it is imperative for the DOE and government agencies invest more in security to protect vital data and assets. While it is still necessary for the DOE HPC centers to continue to tackle traditional threats such as viruses, Trojans, infected attachments, etc., insider threats require a whole new classification of threat deterrence solutions. A broader impact of the proposal is changing the game in favor of defenders and against the insider threat criminals. Insider intrusion activity poses a serious risk to the network security and compromise integrity, confidentiality and availability of the HPC systems and network resources. Existing insider threat detection systems are I/O and CPU-bound with limited scalability and realtime capabilities due to their large data mining and computing requirements. Current insider analysis practice faces several challenges: (a) enormous amounts of data that need to be analyzed in a timely manner, (b) too many false positives in purely structural anomaly detection, and (c) the current lack of automatic semantic interpretation of the data at hand. Statement of How this Problem or Situation is Being Addressed: Cyber Equations Inc. plans to develop a scalable and realtime HPC cybersecurity monitoring system called “CyberInsider” for efficient detection and mitigation of evolving insider threats. To accomplish this, we will develop a framework to systematically explore the space of insider threats and develop a general threat taxonomy for business and government systems. To characterize, track, and mitigate insider threats, we will develop an integrated defense system against diversified cyber threats, which span three defense strategies: proactive defense, reactive defense, and predictive defense. This proposal will implement an Hadoop based hybrid system that integrates and parallelizes anomaly based-machine learning intrusion detection with signature-based intrusion detection techniques. The techniques are divided into five cyclical and parallel processing stages. In the first stage, Deep Packet Inspection (DPI) is applied to the network packet data payload and header (PCAP data), signature-based SNORT is used to generate alerts for anomaly data, and feed network sensors with new generated rules. In the second stage, data mining techniques on Netflow data is performed using unsupervised “k-means” to cascade k-means clustering and regression trees for classifying normal and anomaly activities. The third stage, CyberInsider uses user behavior baselining to detect anomalies in user behavior, and based on detected behavior anomalies create new SNORT rules. The fourth stage, supervised learning will be accomplished using a Support Vector Machine (SVM) for classifying normal and abnormal activities, and decision trees to model decisions, and their possible consequences, and to understand what the clusters (and their respective classifiers) stand for. Finally, using Hadoop we generate a scoring model based on the correlation of DoE security policies, detected signature-based events, and evaluated statistical events derived and from the previous stages. The final risk scoring stage produces accurate and automated true positive insider threat detection. Combined with the aforementioned tasks, a real-world HPC testbed will be developed to validate the effectiveness of the proposed schemes. Commercial Applications and Other Benefits: There is a longstanding problem of threats coming from inside government and large organizations where respected employees become malicious, giving away confidential information and use stolen information for financial gain. These threats happen seemingly without advance notice and cause severe consequences. However, in retrospect, there is often a pattern or trail before the fact that could be traced and uncovered. There are many novel technologies for detecting malicious insider behavior. Such behaviors are relatively rare in the broader user population, and so techniques for anomaly detection can be applied. For example, using machine learning to recognize malicious intent in information gathering commands, detects anomalies in document accesses and queries, and models user processes and flags deviations from the model. There are also many commercial tools for detecting malicious insider behavior through monitoring network activity and the use of enterprise applications. Despite these tools, the incidence of insider attacks continues to rise in the government and commercial sectors. As an example, a recent survey found that 28% of respondents would take sensitive enterprise data to negotiate a new position in the event their employer terminated their current position. While these tools can accurately identify known attacks, they are necessarily reactive (as opposed to proactive) in their enforcement, and may be eluded by previously unseen, adversarial behaviors. In this proposal, instead of investigation after the fact, we seek the capability to proactively identify malicious intent before the intent is carried out. We envision that the proposed techniques, tools and software to have a significant impact on the cybersecurity enhancement of HCP systems. In addition to the security enhancement, the result of this development effort can be extended and tailored for HPC systems in other government agencies and industry to identify insider threat activity.