Heuristic-Aware Anomaly Detection (HAAD)

Cyber Network Operations is a critical new battlefield that holds asymmetric threats to U.S. military, technological, and economic dominance. 21CT’s Heuristic-Aware Anomaly Detection (HAAD) approach develops new behavioral threat detection algorithms that provide fast, effective, flexible, and adaptive defense. Many existing techniques rely on hardcoded signatures, making them brittle, expensive to update, and prone to fighting “yesterday’s war.” Existing anomaly detection techniques lack the ability to detect subtle changes in communication structure or leverage expert knowledge. HAAD leverages two mature network activity anomaly detection and context-aware anomaly detection approaches. HAAD provides flexible and effective defense by extending clustering, dimensionality reduction, and anomaly detection to incorporate heuristic knowledge. Unlike hardcoded signatures, heuristics guide detection without lowering sensitivity to previously unseen attacks. Heuristics can come from expert input, automated conversion of policies, or anecdotal examples of good and bad behavior. This novel approach enables richer and more adaptive behavioral models that improve detection, reduce false positives, and let the system tune itself with minimal intervention. Phase 1 conducts quantitative experiments to demonstrate feasibility and recommends a deployment architecture. The work leverages our long history of developing novel behavior analysis algorithms and detailed knowledge of USAF cyber infrastructure gained under ongoing operational efforts. BENEFIT: The Phase 1 technical effort will address core technical challenges and generate early prototypes, paving the way for full implementation in Phase 2. This provides technical innovation and state-of-the-art research at reduced technical risk. Phase 1 will provide a clear determination of feasibility, along with quantitative evidence to support it. HAAD performance will be quantitatively compared to a non-heuristic-aware baseline on representative test data, using industry-standard performance metrics. Each element of our approach provides important benefits over existing technology, starting with the underlying representations. Graphs are a natural fit for modeling network structure and activity, and they enable the use of both strong theoretic analysis and efficient algorithms. Combining graphs with pattern classification by using Social Network Analysis (SNA) metrics lets HAAD detect subtle activity changes that are not detectable with traditional network metrics. Soft heuristics guide the search without limiting it. Anomaly detection reduces the system’s reliance on large pattern libraries, lowering both the workload and cost of a deployed system and improving adaptation to future needs. Novel extensions of clustering, dimensionality reduction, and anomaly detection, when combined with novel representations, allow the re-interpretation of the same activity in the context of different heuristics. This incorporates expert knowledge without relying on cumbersome knowledge engineering processes. Ultimately, these technical benefits will bring tangible operational benefits to the warfighter. 21CT has achieved this in earlier work by transitioning SBIR-funded technical advances into ongoing operational efforts. HAAD provides benefits to both cyber warfighters and traditional warfighters by securing our networks, which are increasingly critical both to wartime operations and to maintaining our technological, economic, and military edge.