Causation Attribution of Network Traces Across Complex HPC Infrastructure

Highly performing networks are absolutely vital to people, companies, and researchers and even short disruptions can affect millions. Outages at HPC facilities can slow down vital research in vaccines, cancer, weather and physics, and artificial intelligence. When network impairments or cyber-attacks occur, experts and researchers rely on network trace analytics to detect, diagnose and resolve the issue. Even small timing errors lead to inaccuracies slows down root cause analysis (RCA) and increases mean time to repair (MTTR). The problem is harder as HPC networks are more complex. The proposed effort will demonstrate the feasibility, performance and accuracy of an innovative timestamp synchronization method that makes it possible for the first time to realign network traces taken at multiple points in and between complex datacenter and HPC networks. The network will be emulated and accuracy of the network metrics of latency, loss and throughput will be measured by comparing to baselines. The proposed effort will implement novel technology that improves network performance, cybersecurity and troubleshooting. An innovative timestamp method will be implemented and demonstrated in a test network emulating complex datacenter and HPC installations and handle network traces from multiple datacenters and geographies. The result is attribution of network impairments or attacks to the precise point and time it occurred, measure to the microsecond. The Network Performance Monitoring and Diagnostics (NPMD) market is $2B growing 10% annually, other fast growing markets that will benefit causal attribution including Application Performance Monitoring at $5B and CyberSecurity at $250B. The proposed Causal Attribution technology lead to productivity and efficiency gains form HPC supercomputing to telecommuting. Improved productivity for employees working from home, tele-education and telemedicine. Potential subsequent efforts would build upon the Phase I success to develop a full-scale demo and beta deployment (Phase II) leading to a fully commercialized Software as a Service (SaaS) and on-premise offerings (Phase III). If carried over into a phase II, the proposed project will result within 12 months in commercial on-premise and Software as a Service products capable of forensic and real-time analysis of multi-point network traces taken over Complex HPC Infrastructure.