Efficient Distributed Collection and Analysis of Packet Delay Data for Secure Role-Based Export and Topological Mapping

Networked applications work very badly when delay variation (queuing delay) gets large but evidence shows that current queuing delays are barely tolerable and rapidly increasing. There are a plethora of commercial, home-rolled, open source, and university created tools, but none are capable of measuring and monitoring delay variation in real-time or isolating its root cause, thus this projects aim is to fill that gap with no or minimal impact (additional traffic) to the network. The goal of the proposed work is distributed network monitoring to identify delay bottlenecks, process the data to compactly represent its real-time variation, then securely exchange the results in a way that allows all the involved parties to collaboratively locate the delays root cause while not compromising their privacy or security. Making the secure, privacy-preserving dissemination of the new delay information part of the solution allows it to be easily incorporated into existing network management products. The project plan includes working with established small businesses in the network management area to integrate delay-based diagnostic information and working with linux developers to integrate the measurement points into the kernel. A method and proof-of-concept software implementation were developed for a Transport Segment Delay Estimator (TSDE). The TSDE infers path delay behavior based on a novel extraction of information collected at a single point. Using simulated scenarios, the method was successfully validated and output data was compressed by three orders of magnitude. A prototype of the TSDE will be implemented in a commodity network device and tested. A packet loss detection feature for the TSDE will be evaluated. A secure, distributed measurement architecture to include TSDE results in a Network Management Aggregator will be developed, using standards-based approaches where possible. Commercial Applications and OtherBenefits: The TSDE can be deployed in home or ISP routers. The data can be used by network managers to detect and isolate bufferbloat, a drag on productivity and a barrier to energy-saving programs like telecommuting and telepresence. TSDE data can pinpoint most critically affected parts of infrastructure. The techniques can be used in the the DoE networks and between networks to share data.