SBIR Phase I: Cloud computing for real-time processing of photon light source data

This SBIR Phase I project proposes to use commercial cloud computing resources for real-time processing of X-ray computed tomography data collected at a photon light source facility. The amount of data being collected at photon light sources is poised to increase dramatically due to improvements in detectors for more rapid data collection and advancements in the underlying accelerator technology for greater production of photons. As a result, storing and processing data sets on a scientist's personal workstation or small computing cluster will become impractical as data set sizes continue to grow much faster than Moore's law. The proposed solution is to develop a persistent, managed computing cluster in the cloud that can grow to meet the computational and storage needs of scientists without them having to deal with managing physical hardware. X-ray computed tomography is used as a test application where an automated workflow can be developed that will store and analyze the collected tomography data during the experiment with minimal user intervention. The broader/commercial impact of a persistent, managed computing resource in the cloud is providing scientists an affordable tool that can grow to meet their computational and storage needs without the hassle of managing physical hardware. Storing and analyzing data are routine tasks that are fundamental to scientific research. As technology improves data collection rates and data fidelity, scientists are dealing with larger and larger data sets. Some examples of this growth include genomics, where the cost to sequence a genome has dropped precipitously, astronomy, where powerful telescopes can quickly capture high resolution images, and high energy physics, where data sets need to be reduced in real-time so that their sizes are more manageable. This dramatic increase is contributing to the "Big Data" problem that scientists are facing as their traditional data processing tools are no longer sufficient to tackle these data sets. By taking advantage of the economies of scale achieved by cloud service providers, Billow can provide reliable storage and high availability computing at a lower cost. This approach allows scientists to focus on the science by giving them the necessary tools to analyze their data and develop new data analysis algorithms.