SBIR Phase I: Scaling Cluster Computing Fabrics with Optical Networking

The broader impact/commercial potential of this Small Business Innovation Research (SBIR) Phase I project will result from developing technology to enable enterprise, cloud, academic, and government network operators to cost-effectively scale computer clusters to support the growing demands on these systems. Computer clusters underpin an increasingly broad range of services that businesses, researchers, and society have come to rely on, from machine learning systems to online social media platforms. Companies increasingly use computer clusters to gain a competitive advantage and accelerate product development. Today's networking technologies improve in performance by a factor of two every two years, but the demand placed on cluster computing systems is growing more than twice that fast in some market segments. The technology developed through this project is well-positioned to exceed the cost and performance scaling limitations of current networking technology, enabling business and scientific end-users to find solutions to more complex problems more quickly. This Small Business Innovation Research (SBIR) Phase I project demonstrates the feasibility of a network interconnect for computer clusters that routes traffic through transparent optical switches. Optical switches remove communication bottlenecks in the network fabric but require fundamental changes in how distributed applications communicate information across the network compared to present-day technologies. The commercial success of the technology depends on its ability to accelerate the execution of common distributed applications while providing the interoperability, reliability, and manageability expected by network operators. This project aims to demonstrate a 2x improvement in the execution speed of network-bound applications compared to today's technologies. The proposed research to reach this goal includes developing and optimizing the software stack necessary to interface commodity applications with the fabric, quantifying the performance of those applications subject to the constraints imposed by optical switching, and optimizing the optical switch design to maximize application performance at a system level. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.