DIA: Distributed Information Assurance in Ballistic Missile Defense Systems using Scalable, Distributed Data Mining Technology

This document proposes to develop a Distributed Information Assurance (DIA) system based on the distributed data mining technology for detecting distributed network attacks and identifying attackers’ “signatures” for advanced situational awareness. It will offer the following key capabilities: 1. A multi-agent architecture for linking multiple, heterogeneous network-sensors (e.g., intrusion detection and malware detection systems, netflow data, tcpdump) for performing distributed and decentralized analysis of the data. The system will support local management of policy-based control for different sensors. 2. A collection of distributed data mining algorithms for decentralized outlier detection, clustering,and trend analysis for network data. These algorithms will lay the foundation of the DIA system. 3. A module for supporting the complete life-cycle of the information assurance management process in a BMDS. Following attack detection capabilities will be explored during Phase I: Detect distributed network “signatures” of attackers based on the distributed observations from different nodes in the network. Detect attack patterns on the coalition members in terms of clusters, outliers. Identify statistical properties of attack distribution in order to perform a trend analysis. Detect stealth network probes by attackers and worms. The proposed work will be performed at Agnik, a mobile and distributed data mining company.