Network Measurement and Congestion Analysis

Network operators, within and outside the Federal Government, gather a large amount of data about their networks using sophisticated collection tools- In particular, the Department of Energy operates many heavily- trafficked networks, and has done an excellent job implementing processes and developing applications which collect significant amounts of network measurement data- Unfortunately, data analysis often lags; most collected data sits unused, unable to help network operators improve quality of service or glean insight about network performance- When some of this data is excavated and inspected, typically during a network incident postmortem, analysis is laborious and often done by hand- Current network tools exclusively use static signature- based algorithms to inspect this type of network data; the current proposal uses dynamic signatures arising from the data itself to make sense of complex network interactions- The current lack of sophisticated automated data processing devalues the existing data collection infrastructure and fails to extract insight from or otherwise take advantage of immense quantities of assiduously harvested BigData- The proposed work creates software for the detection of congestion conditions/anomalies in data streams collected from a network in the form of time-series data- The software will be configurable to alert for smaller or larger congestion conditions as some network operators might only want to be notified for major events, while others may want to receive numerous notifications daily-