Technologies for Sharing Network Performance Data (TEECODE)

Government and commercial sectors need high-performance secure networks to carry out their missions and operations. High-performance networks that connect the DOE National Labs and their scientific facilities and scientific and academic institutions within and outside the nation are extremely critical to fulfill the mission of enabling scientists nationwide and globally to perform and deliver uninterrupted scientific discoveries. There is a critical need for tools that help manage and improve the performance of networks and these tools must correlate different forms of network data to present a unified picture of the network infrastructure and enable the network administrator to get holistic insights into network health and performance to take proper actions. We propose TEECODE as a solution that provides an effective, usable, and scalable correlation tool to bind multiple network data ensembles together and present actionable insights and holistic knowledge on the network to the network administrator. The approach used for deriving and presenting correlations to enable network visibility will involve an unsupervised machine learning technique for data analysis that is based on an advanced mathematical tool that can effectively extract coherent patterns and correlations in data. The proposed approach will result in an easy-to-use tool that is scalable to handle large volumes of diverse data ensembles that may be anonymized for security reasons and seamlessly provide visibility into network state and performance. In Phase I, we will develop a proof-of-concept prototype of a usable and scalable correlation analysis pipeline that can scalably and flexibly extract, transform, and load diverse network data ensembles, analyze the data ensembles and extract correlations, and present the results of the correlation analysis in an user-friendly form to the network analyst. We will extensively test and validate the solution on real network data. We will demonstrate the solution to potential early adopters and lay the groundwork for building a mature product and commercializing the solution. TEECODE will significantly help improve the performance, reliability, and operation of commercial, research, and Government (DOE, DoD, and other agencies) networks, and will help reduce the overall operational costs and risks of network operation. TEECODE will bring tangible benefits to commercial sectors including, but not limited to, telecommunications, healthcare, insurance, and finance sectors, by offering improved visibility, reliability, and protection to the networks that are critical for their operations and business. TEECODE will have a very positive impact on the scientific community and enable them to fully utilize the sophisticated infrastructure provided by high-speed high-performance networks, conduct high-end research, and discover new scientific innovations. Increasing the power of scientists by enabling them to do uninterrupted discovery will directly contribute to national and global prosperity.