Correlate Log Data and or Host Sensor Data with Network Trace Data

Log data collected by hosts and servers provides a high-level view of how these systems are being used. For example, GridFTP server logs provide details on what files were moved and how the service decided to perform the transfer task (e.g., parallel or sequential processing) of files. This data contains local and remote host names, which trivially translate into IP addresses. It is important that this log data be correlated with the network trace data in order to diagnose network performance problems and issues. Without this correlation it is difficult or impossible to fully explain why transfer times differ from one download to another. Oceanit has developed a powerful technique that correlates different modalities of the same phenomena such as matching a textual label to a bitmap image of an aircraft. In this project, Oceanit will apply this technique to correlate textual server logs with binary network trace data. Oceanit has already used this technique successfully to label dynamic execution traces for malicious program understanding and to document Windows binary dependencies for software supply chain vulnerability identification. For this effort, Oceanit will leverage over $2M in funding from the Army, Navy, and Homeland Security. Oceanit plans to apply its powerful technique that links different modalities of the same phenomena to correlate log data and host sensor data with network trace data. In Phase I, Oceanit will demonstrate the correlation (matching and scoring) of logs from multiple services, applications, and processes with corresponding networking trace data. Network management is critical for many businesses and plays a crucial role as part of normal operations. Any breakdown or disruption that slows network performance will adversely impact the quality of service leading to outages of applications many employees rely on including virtual meetings and cloud-based platforms. Our tool will facilitate diagnosing networking issues and help engineers rapidly fix problems by allowing them to fully understand the fundamental causes of the problem.