SBIR/STTR Commercialization Assistance - Ease of use and deployment for a fast, scalable data movement infrastructure

In recent decades, High Performance Computing (HPC) has risen as one of the key enablers of innovation in science and engineering. However, complexity and cost of deploying applications impedes industrial adoption. Problems include the quality, speed, and flexibility of software tools and the specialized expertise required to adapt solutions given the profusion of hardware configurations. Efficient data reading and writing (I/O) remain a key bottleneck in standard approaches, often limiting the scalability of application codes, reducing the scale and scope of problems that can be addressed with simulation. How the problem is begin addressed: This SBIR/STTR Phase II will support the deployment of PIDX, a high-performance parallel I/O library, which scales efficiently to a million cores on DoE flagship HPC systems. The key innovation enabling high-performance I/O is a proprietary hierarchical space-filling data layout that delivers optimal performance for memory hierarchies. PIDX and the ViSUS framework enables fast visualization and analysis, even for petabytes of data stored on remote systems. This SBIR/STTR will make this technology accessible for commercial use by reducing the time and complexity of installing and using the PIDX library and ViSUS framework. Phase I results and Phase II course of action: Phase I hardening and deploying the PIDX library in the Uintah project resulted in reducing the I/O cost from 49% to 2% of the compute time, effectively dou- bling scientific productivity. With a very conservative estimate of just 2 cents per core compute hour, this simulation was completed with over $3M in savings for the DOE, already returning more than double the investment costs for the proposed Phase II SBIR/STTR grant. This SBIR/STTR Phase II will continue to harden the existing PIDX library and deploy performance improvements. The PIDX library will be lever- aged to enhance post-simulation analysis and visualization workflows. Finally, the breadth of simulations accelerated by PIDX will be expanded by developing support for adaptive resolution meshes. Commercial impact and societal benefit: A simple and efficient I/O ecosystem is a prerequisite component of engineering and industrial use of HPC systems. Removing I/O bottleneck will enable more efficient use of expensive HPC resources, allow larger and higher-fidelity simulations for more accurate modeling, and reduce the cost and complexity of writing scaling simulation codes. The data addressed by this project is already being produced and utilized commercially in reservoir simulations in oil and gas exploration, fluid dynamics studies in aerospace industries, and computational electromagnetics in manufacturing processes. The industrial HPC market saw a 10% growth in the first quarter of 2015, which is further expected to expand as additional applications are adopted. The I/O library supported by this SBIR/STTR will furthermore ease the burden of scaling, as today’s scientific research capacity runs become tomorrow’s commercial capability processing.